Python: justification for boolean operators (and, or) not returning booleans

[EDIT: this question is not a duplicate of this; I am asking a question of language design, i.e. why it is this way in the first place. I am not confused about how it behaves, I am confused about why this behavior was decided upon in the first place.]

In Python (and some other languages, e.g. Lua), the boolean operators and and or do not return boolean values True and False as one might expect, but rather:

• x or y: if x is falsey, then y, otherwise x
• x and y if x is falsey, then x, otherwise y

It is easy to see that in the special case where x and y are booleans this behaves as expected.

My question is, what is the actual justification for generalizing in this way? I know that you can use it for tricks like:

foo = x or None

Or a DIY ternary operator like:

foo = x and y or z

But I don’t feel as though these are strong enough justifications for such a surprising behavior. Whenever I see the non-boolean return value being relied on for its actual value (rather than in an if statement or something, where it doesn’t matter either way), I always have to double-check my understanding of it since I always forget how it works and I need to look it up again. I would usually prefer it written in a more verbose way. Is there a fundamental reason why it should work this way, or is it really just for tricks?

Answer

In Python’s case, the bool type didn’t even exist until over a decade after the language was released. If it had existed from the start, perhaps and and or would have been defined differently. But, as is, backward compatibility was far more important. From the PEP that introduced the type:

Another consequence of the compatibility requirement is that the expression “True and 6” has the value 6, and similarly the expression “False or None” has the value None. The “and” and “or” operators are usefully defined to return the first argument that determines the outcome, and this won’t change; in particular, they don’t force the outcome to be a bool. Of course, if both arguments are bools, the outcome is always a bool. It can also easily be coerced into being a bool by writing for example “bool(x and y)”.

EDIT: BTW, at its start, Python was intended to “bridge the gap” between programming in “data-rich” languages like C and “convenient” languages like Unixy shells (sh, csh, etc). Python’s and and or were more like && and || act in shell languages, consuming – left to right – all and only the operands needed to resolve whether the overall chain succeeded (“true”) or failed (“false”).