The yellow most likely builds up slowly, as sweat and skin oil left in the fabric age over many wears and washes. The smell stays because skin bacteria can survive the wash on the fibers.
A fresh sweat mark is invisible. According to a University of Kentucky extension guide, it "can oxidize and turn yellow with time" (source). While you wear a shirt, sweat and skin oil pass into it along with bacteria from your skin (source). In a lab test, fabric soiled with squalene, a common skin fat (source), turned yellow as it aged, and laundering didn't fully remove the yellow (source). That was a model study, and an older test found two other skin fats didn't yellow cloth by themselves (source). So skin oil is a strong lead, and probably not the only cause.
Washing can leave some of it behind. In a study across six university labs, white cotton soiled again and again with artificial skin oil and clay built up soil over 24 rounds, and hotter water removed more of it (source).
Products with aluminum salts may add to the problem. One conference paper says that with sweat they leave yellowish stains, and "These stains are very hard to remove with regular machine washing" (source). It's a single paper, funded by a washing-machine maker, and it didn't compare against stains made without aluminum.
The smell comes from bacteria breaking down the sweat and skin oil in the fabric. Researchers found that "When the biofilm persists after washing, the textiles retain the sweat odor" (source).
The same Kentucky guide advises washing "using the hottest water safe for the fabric or garment" and says "Stubborn stains frequently respond to washing with an enzyme laundry detergent or oxygen bleach." Check the care label first. And "Don't put clothing in the dryer until all stains have been completely removed" (source).
Treat the pits first, and save the dryer for last.
Several of these are small or lab studies, and we say so when we cite them.
Déodorant N°1 is our fragrance-free deodorant cream, made for people who train. 100% stink-free in 30 days or it's free. Shop Déodorant N°1