His previous coalition ally, the far-right Freedom Party (FPÖ), and the opposition Social Democrats (SPÖ) on Monday backed the no-confidence motion.
The FPÖ had become embroiled in a political scandal caused by a secret video, which ended the coalition.
On Tuesday, Austria’s president appointed an interim government led by Vice-Chancellor Hartwig Löger.
President Alexander Van der Bellen earlier said the constitution mandated all offices must be filled “even in a transitional period” and asked for some ministers to stay in office. General elections are expected in September.
Mr Löger is a member of Mr Kurz’s centre-right People’s Party (ÖVP) and the country’s finance minister.
He was appointed vice-chancellor days ago after the previous holder of the office, FPÖ leader Heinz-Christian Strache, was sacked over the video sting.
Mr Kurz is the first chancellor in post-war Austrian history to lose a confidence vote.
When he was elected in 2017, he was at 31 the world’s youngest state leader.
What happened in parliament?
Opposition parties brought forward two no-confidence votes – one against Mr Kurz and the other against the government.
While the SPÖ control only 52 of the 183-seat lower house, the FPÖ – which holds 51 seats – also backed the motions, which needed only a simple majority to pass.
The left-wing environmentalist JETZT party voted to oust the chancellor and his government, although the liberal NEOS party reportedly backed Mr Kurz in a bid to avoid instability.
Mr Kurz’s surprise strong showing in Sunday’s European Union elections – winning a record 35% of the vote – was not enough to save him.
Speaking after the confidence vote, Mr Kurz pledged to support an interim government and insisted he and the ÖVP had “guaranteed stability” in Austria.
The parties ranged against Mr Kurz appeared to believe he should shoulder some of the blame for the fall of the coalition.
The Social Democrats said he should never have allied himself with the FPÖ in the first place. The FPÖ was still smarting from having had Mr Kurz replace all of its ministers with technocrats.
The video sting has widely been labelled “Ibiza-gate”, after the Spanish island where the footage was recorded.
It was secretly filmed in July 2017 – just weeks before the election which saw both the FPÖ and Chancellor Kurz’s party perform well.
Last week, a Vienna lawyer said he had been involved in the sting, describing it as a “civil society-driven project in which investigative-journalistic approaches were taken”.
However, the lawyer did not reveal who was ultimately behind the operation, or who paid for it.
In the footage, released by German media this month, Mr Strache can be seen relaxing and drinking for hours at a villa with FPÖ parliament group leader Johann Gudenus, while they meet a woman, who says she is Alyona Makarova, the niece of Russian oligarch Igor Makarov.
Mr Strache appears to propose offering her public contracts if she buys a large stake in the Austrian newspaper Kroner Zeitung – and compels it to support the FPÖ.
He is heard suggesting that a number of journalists would have to be “pushed” from the newspaper, and that he wants to “build a media landscape like [Viktor] Orban” – referring to Hungary’s nationalist prime minister.
It later emerged that Mr Makarov does not have a niece, and the whole evening back in 2017 was an elaborate trap.
Mr Strache stood down hours after the video emerged.
President Van der Bellen then fired FPÖ Interior Minister Herbert Kickl at the request of Mr Kurz.
The move prompted the FPÖ’s other ministers to resign in solidarity.
Despite the scandal, Austrian news agency APA reports that Mr Strache could possibly take a seat in the European parliament.
The former vice-chancellor had remained at the bottom of his party’s election list for the European elections after his resignation. But under Austrian law he could take one of FPÖ’s predicted three seats if enough people supported him as a candidate.
It is unclear whether Mr Strache will take a seat.