Jerusalem (CNN)-- Naftali Bennett was sworn in as Israel's new prime minister on Sunday, after winning a confidence vote with the narrowest of margins, just 60 votes to 59. His victory ends a 12-year grip on power by former prime minister Benjamin Netanyahu, the country's longest-serving leader.

After four elections in two years, Bennett's incoming government breaks a long political deadlock and ushers in the most diverse coalition Israel has ever seen, including the first Arab party to serve in the government. In his speech before the Knesset confidence vote, Bennett celebrated the diversity and warned of polarization within the country.

"Twice in history, we have lost our national home precisely because the leaders of the generation were not able to sit with one and another and compromise. Each was right, yet with all their being right, they burnt the house down on top of us," Bennett said. "I am proud of the ability to sit together with people with very different views from my own."