The best way to start is with the Complete Symphonies (BIS 843, incl. Sinfonia, and In Memoriam). If that’s too much at once, try Symphonies nos. 8 and 9 (BIS-618) or Symphonies nos. 6 and 7 [HA23] (BIS-573[HA23]). I would not want to miss any of the gems in the chamber concerti series, either. It makes sense to follow the string quartets in their order of issue.
Complete Symphonies nos. 1–13, Sinfonia, and In Memoriam, BIS-843
Symphonies nos. 8 & 9, BIS 618
Symphonies nos. 6 & 7, BIS 573
Chamber Concertos: nos. 1–3, nos. 4–6, nos. 7–9, nos. 10–13, Marco Polo 8.224038, 8.224063, 8.224086 and 8.224087.
Primavera, Ballata, and other chamber music, Dacapo 8.226073
Viola Concerto; Concerto for Orchestra; Second Violin concerto, Dacapo 6220599
Complete String Quartets, Dacapo 8. 207001. This seven-CD set is an extraordinary bargain at less than $35 (on Amazon), though for the cautious, the CDs are available separately.
Toccata Press, the book-publishing arm of Toccata Classics published Experiencing Music, an invaluable book of writings by Vagn Holmboe.