And here's a differing POV:

https://www.rollingstone.com/music/migos-drake-future-albums-long-streaming-w516991

Even after the LP was officially dropped – it was a Phil Collins release that was the first non-LP major act, from memory – the pop CD in particular was not over filled to capacity; it’s only really been a recent phenomenon, as RS is pointing out.

From an industry POV, record companies were keen to keep the track count down to minimise paying mechanical royalties to the song publishers there-on – this was particularly true in the C&W world where the LP/CD was on average 10 tracks and no more. Classics were different – it was rumoured that thje initial max of 74 mins was landed on because Herbert Von Karajan insisted that Beethoven’s 9th should be complete on one CD – given that Sony and Philips developed the CD, and HvK had nowt to do with them at the time, suspect it’s an urban myth. 30 mins was the tolerance for one side of an LP, and that was pushing it for symphonic music.

If Bob Dylan conceived a new “Blonde on Blonde” today, how long would it run, I wonder? (“SELOTLL” was cut slightly to fit the 2-lps on one CD.) Is there filler on The White Album? Runs 94 mins, about peak for pop lps the. Would they have been tempted to go for 160 mins as a double-CD or cut out Ringo’s contributions and fit it on one?

Suspect the point about rap is that it can mostly be recorded almost live – Kanye West etc. notwithstanding – which is why we have a surfeit. Despite increased technology these days, a work like “Sgt Pepper” or “DSOTM” would still require a lot of studio time…

These and other questions will likely never be answered.