He was born and brought up in Neutra where his father was hazzan and Goldstein sang in his choir from age six. His father died when Goldstein was ten and when he was 13 he took his place as hazzan of the community. He officiated in many Hungarian communities and studied in Prague before being appointed at age 18 chief cantor at Vienna’s Leopoldstadt synagogue where he remained for over forty years. He introduced there the Polish style of hazzanut.