Mammary gland growth is an important phenotype due to its correlation with future lactation performance. Currently, biopsies represent the standard procedure to procure histological data to assess mammary gland development. However, analyzing individual microscopic images is a labor-intensive and time-consuming task that can only provide a few visual and manual measurements, limiting the ability to extract meaningful or novel phenotypes. Computer vision has been a powerful tool in medical sciences for real-time identification and segmentation of biological tissues, as well as quantitative analysis on tissue size, and geometric features. Therefore, this study aims to implement a computer vision system for the multilabel semantic segmentation of microscopic images from biopsies of the mammary gland using a U-net model. The segmentation model was trained on over 80,000 samples, as a result of patching 80 high-resolution histological tissue biopsies from 20 dairy calves at 10, 26, 39, and 52 wk of age, and tested on 56 images (56,000 patches from 14 animals). Tissue growth was analyzed with a linear mixed model including diet, week, and their interaction as fixed effects, birthweight as a covariate, and animal as a random effect. The segmentation model achieved an intersection over union of 0.88. The correlations between predicted and observed adipose, stroma, and duct area in the testing set were 0.99, 0.99, and 0.97, respectively. There was a significant effect (P < 0.03) of week on adipose and stroma area (% of total issue), ducts and adipose circularity, and ducts and stroma eccentricity. The adipose tissue area (%) increased 5 times from wk 10 to 26 and 1.34 times from wk 26 to 52. This suggests a nonlinear and decelerating growth development of the mammary gland during the period evaluated. Although the association between such rate of growth and lactation performance should be investigated, the use of image-based features can allow for large-scale phen