Difficulty isn't really a concern for me. The "narrative" of the puzzle is way more important - by which I mean the wonder & confusion when you first see the puzzle, to the roadblocks you hit when you try the obvious stuff, to the satisfying eureka moment when you solve it. The Witness does this by chaining 5-10 puzzles together, so it's really smooth, but feels kinda diluted by the sheer # of panels. Baba is the absolute master of this, the mental journey that every single level takes you through is masterful. Portal totally masters puzzle narrative too, but with much easier puzzles. Other great examples are Monster's Expedition and Void Stranger.
Sausage was a little too rough with its puzzle design so you hit walls constantly. Snakebird was similar, so even though its puzzles were easier, I bounced off of it even faster. Cocoon is so smooth that it ends up being totally flat, so you end up at the solution without having to think or actually solve anything.
In this regard, puzzle design is no different from other level-based game design. Celeste's levels have the same kind of eureka-based narrative flow, and so I don't think the enjoyment of Celeste is particularly different from Baba. It's just that one uses action mechanics and one uses sokoban mechanics. Unless you're physically bad at one of those mechanics, you should be able to enjoy both about equally.
That said, you kinda need difficulty to deliver the strongest eureka moments. It's like hot sauce - they're all delicious, but if you like spicy food, you're gonna need to keep escalating the heat to get the same enjoyment. That's why I can't say a game as easy as Portal is the "best" puzzle game. But if you think ketchup is spicy, you're gonna like Cocoon a lot more than Baba, and Mario Wonder more than Celeste.
This post sums it up perfectly, and better than I could have.
I love that you compared Baba to Celeste, because they are both in my all time top 10, and your reasoning is dead on.