And the solution is not at all intuitive. Here's what you need to do (solution taken from here):
- Region and Language Settings
- Additional Date, Time & Region Settings
- Change input methods (under Language)
- Advanced Settings (Left menu/options)
Then click on "Apply language settings to the welcome screen, system accounts and new user accounts", under section "Override for Windows diplay language".
Sorry providing the walkthrough in a textual way. Too much screens to screenshot this time...