For the time being, I found a registry hack, hopefully this will be included as a setting in one of the future updates:
Go to: \HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Shell\Update\Packages and create new 32bit Dword Value, name it UndockingDisabled give it the value 1.
Important!!! Please note: this will reset the taskbar to "classic style" and unfortunately it also disables the Start button and disables the taskbar clock. In addition to that, it seems to make logon/logoff significantly longer and Windows key functions like (win+v or win+shift+s) will stop working.
This can be fixed though:
Installing e.g. Open-Shell will get you a start button (open-shell will reinstate the classic Start button as used in Win7 and earlier). Alternatively, you can switch to Windows 10 Start button, but I couldn't get this to work. You can also use third-party tools to get function like win+v (clipboard manager) or win+shift+s (taking screenshots).
To get the clock back use the Control Panel Applet for the notification area by opening explorer shell:::{05d7b0f4-2121-4eff-bf6b-ed3f69b894d9} via cmd and enable the system icons in the option Turn system icons on or off. In addition to the clock you will be able to add individual items like sound control etc.:


Go to: HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies and create new key: Explorer. Create new 32bit Dword Value, name it NoTaskGrouping give it the value 1. Note: this will disable grouping of task icons
Optional: HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Search and create new 32bit Dword Value, name it SearchboxTaskbarMode give it the value 0. Note: this will enable/disable the search icon next to the Start button (which doesn't work with UndockingDisabled anyway).
Sign out and back on (restarting explorer is not enough)
Note: this disables the new Win11 design/functionality for the taskbar and the Start button. I'm still waiting for a better solution, so I didn't accept this as an answer.
Note: I still haven't found a way to get the Start button back without third Party tools. Same for the Windows key functions like (win+v or win+shift+s) but everything else works without third Party tools.
Conclusion so far: Although this is a potential way to ungroup the taskbar without 3rd Party software in the future, it does not quite work yet (at least I couldn't get it to work). So for now, since you need 3rd Party software anyway, I would recommend installing valinet/ExplorerPatcher from GitHub it basically creates the same outcome (Taskbar changes to Windows 10) but you'll keep the Windows 11 Start button as well as the Windows key functions like Win+V, etc. (see my other answer)