Zundel was born on the first day of Rosh Hashana in 1786 in Salantai, Lithuania. Little is known of his early years. He descended from Rabbi Faivush Ashkenazi of Vilna (late 17th-early 18th century) and his father was Rabbi Benyamin Beinush, who was a shochet and hazzan in Salant.
Salant's wife was Rochel Rivkah, and they had three children, two daughters, Tziviah and Heniah, and an only son, Aryeh Leib. Rabbi Yosef Zundel of Salant refused to accept any rabbinical positions. He ran a small business which produced only a meager living. He chose to spend much of his time immersed in Torah studies and musar.
Zundel provided the spiritual inspiration for his most famous student, Rabbi Yisrael Salanter, the founder of the Musar movement.
During the early years of the Musar movement, Reb Zundel was seen in the marketplace on Friday afternoons reminding the merchants that the Jewish Sabbath was approaching so they had time to close their stalls and avoid its desecration.
Rabbi Yosef Zundel, who was a student of the Vilna Gaon in every sense of the word, longed to settle in the Land of Israel. Finally, in 1838-39, despite the hardships of such a trip due to the ongoing war between the Ottoman Empire and Egypt, Rabbi Zundel nevertheless took his family and traveled to Jerusalem. The Ashkenazi community in Jerusalem at that time was under the leadership and financial support of the Kollel Vilna, whose headquarters were in Amsterdam. It was led by a committee under a wealthy Dutch Jewish banker, Rabbi Avraham Zvi Hirsch Lehren (1784-1853). Rabbi Lehren had, in 1817, assumed the mantle of leadership of an organization founded in 1809 known as Pekidim and Amarkalim of Eretz Yisrael ("Officials and Administrators of the Land of Israel"). This charitable organization was in charge of the collection, administration, and disbursement of all the monies collected in Western Europe on behalf of the Ashkenazi Jewish community in the Land of Israel.
Although Rabbi Zundel was an extremely unpretentious individual, Rabbi Lehren was keenly aware of his greatness and had tremendous respect for him. Rabbi Zundel's knowledge of Torah was extraordinary, and the entire scope of Torah was clearly engraved on his heart and mind. He was also a Tzadik extraordinaire. In 1837, Zundel settled in Jerusalem, where, at the urging of Rabbi Lehren, he served as the rabbi of the Ashkenazi community. For centuries, all halachic disputes and queries in Jerusalem were brought to the Sephardi rabbinical court, which adjudicated them. Due to the recent growth of the Ashkenazi community, Rabbi Lehren wanted Ashkenazim to have independence and to adjudicate disputes in their own rabbinical court. Rabbi Yosef Zundel agreed on the condition that he would not be paid a salary for his services. Throughout his life he had worked to support himself in a manner that did not cause him to benefit from Torah, and he wanted to maintain that practice. Furthermore, he had always shunned all positions of honor and distinction. He also stipulated that as soon as he would find a suitable replacement, he would relinquish his position to the other person. Rabbi Zundel opened the Beis Din, albeit as a temporary court. In time though, both the Ashkenazi and Sephardi communities in Jerusalem recognized Rabbi Zundel's greatness and brought him all their halachic questions and issues.
In 1841, when his son-in-law Rabbi Shmuel Salant arrived in Jerusalem, Rabbi Zundel asked him to join the Beis Din. Not long afterwards, Rabbi Zundel, realizing the vast greatness of his son-in-law, appointed him as the official head of the rabbinical court, a position that Rabbi Shmuel held for almost seventy years until his death in 1909. Rabbi Shmuel held the title of "first Chief Rabbi of Jerusalem."
Rabbi Zundel lived in a small one-room apartment in the Hurva Synagogue complex in the Jewish Quarter of the Old City. Rabbi Zundel sustained himself and his family by selling vinegar, but spent most of the day and night in the Menachem Zion Synagogue, which was completed in 1837. Built by the Perushim (students of the Vilna Gaon), it was named after their leader Rabbi Menachem Mendel of Shklov. Rabbi Zundel lived an esoteric lifestyle and radiated spirituality to the entire community.
Rabbi Zundel was instrumental in the founding of the famed Etz Chaim Yeshiva, the Bikur Cholim Hospital and Hevrah Kadisha, and aided in the absorption of new immigrants in Jerusalem. Rabbi Zundel and Rabbi Shmuel worked together tirelessly for the benefit of the community. In 1860 they founded the Rabbi Meir Baal Haness charity, which became the foundation of support for the entire community at large, and saved it from sinking into a financial abyss. Unfortunately, Rabbi Zundel lived for only a few years after its establishment. However, during the final years of his life, he devoted himself wholly to bringing this monumental undertaking to fruition; upon his passing the Rabbi Meir Baal Haness charity had already become a lifeline for hundreds of poor and destitute families.
During the 1860s, the water supply to Jerusalem was very poor, despite several attempts by the Ottoman authorities to repair the ancient conduit from A'yn Arrub and Solomon's Pools. The stone pipes were regularly sabotaged by the Arab farmers who earned a nice living selling water that they brought in unhygienic animal-skin bags from A'yn Rogel and the Gihon Spring through the Dung Gate. The water was sold at a high price, even though it was of poor taste, smelled foul and was dangerous to consume without boiling it first. This water supply depended mainly on the cisterns dug near or even under the houses, in which rainwater collected. In the 1860s there were almost a thousand of these. This water was only fit for drinking as long as it was not contaminated by sewage water. There was no sewage system in Jerusalem at the time, and sewage often ran in the street, seeping into the wells. The pollution of the drinking water brought about a severe plague, which claimed hundreds of victims, and led to the city being placed under quarantine for four months. Sir Moses Montefiore came to the aid of the inhabitants by contributing money for improving the water supply.
Rabbi Yosef Zundel Salant died due to the plague on Friday, October 12, (3rd Cheshvan) 1866 and was buried on the Mount of Olives. Among the many instructions that Rabbi Zundel mentioned in his will was that no eulogies be held for him, and no titles should be added to his name on the tombstone. Rabbi Yisrael Salanter, the founder of the ethical Musar movement and prime student of Rabbi Zundel, is recorded as having said: "I have not found a true servant of God like my mentor Rabbi Zundel!".