Kaski was born in Pielisjärvi into a cantor's family. His father taught him the violin, and was generally supportive: when Kaski dropped out of secondary school to study music full-time in Helsinki, he did not object. Kaski first studied at the Helsinki Cantor-Organist school, then entered the Philharmonic's Orchestra School. There, he continued studying the violin, but was also tutored in theory (by Ilmari Krohn) and composition (by Erkki Melartin).