I was of the understanding that we should not use mother’s milk for our baby if it came from a mother who consumes Cholov Akum. Is this indeed correct?
Yes, it is correct.
See Rema (Yoreh Deah 81:7) & Shach 81:25 & Taz 81:12.
See for example Igros Kodesh vol. 1 pg. 238 (non-kosher food that a nursing woman consumed produces a negative influence), Igros Kodesh vol. 16 pg. 243 (milk from a Goy brings doubts in our Emunah).