Entirely reasonable question, but I wasn't talking about making custom layouts for each device. I'm also talking about a single UI design that is responsive to different screen sizes. But instead of supporting all iPhones, if we instead support 4.7-inch screens and above, that's less work, because there are fewer cases to handle.
Taking your example of expand/collapse, when you implement it, it's more complex than it appears when watching a Youtube video: you now have to handle three states: a collapsed state on small screens, an expanded state on small screens, and big screens, which don't have expand/collapse. This imposes more work on the designer to design for these states and make sure they're all usable and look good, perhaps with animated transitions between them. On the engineers, who have to deal with multiple code paths and edge cases. On QA, to spend more time trying to test more scenarios on different devices. Everything sounds simple till you code it up :)
Now, if you could eliminate this complexity by not having to collapse the control in the first place, that would be simpler. Maybe there's enough space on the 4.7-inch screen for that.
UIs scale up better than down. I prefer to do UI design for the smallest supported screen size. If we choose to support 4-inch screens, we may have to choose a UI design that's not optimal for 4.7-inch and bigger screens, which the majority of our users actually use. Whereas if 4.7 was the baseline, we may be able to come up with a better design for most of our users.
All these problems are, of course, solvable, but if they take X amount of time to solve, is there something more beneficial you can do in that time?