Yes. Absolutely yes. Just as an American citizen can be arrested and put in jail in any foreign country they visit.
As for whether they would be deported or not, it depends largely on the crime they have committed and whether or not their country of origin would continue the punishment sentenced by the US courts. Generally speaking, the foreign citizen would serve their full term in prison (with time off for good behavior), then undergo deportation proceedings.
Manuel Antonio Noriega Moreno, universally known as Manual Noriega, is a former Panamanian politician and military officer. He was military dictator of Panama from 1983 to 1989 when he was removed from power during the invasion of Panama.
And where they removed him to? Correct!
On September 16, 1992, Noriega was sentenced to 40 years in prison, later reduced to 30 years.
Yes, US prison. And they didn’t deport him.
He wasn’t an American citizen.