Just about everywhere I searched on how to do this suggested to go to your hybrid server onsite and download the Exchange Online Powers hell Module. As you can see in this picture I do not have that option for some reason:

I then found the nice site in the reference and installed the ExchangeOnlineManagement module like so:

Install-Module –Name ExchangeOnlineManagement

I did have to say yes to some prompts since it is from an unknown source however the source does seem to be an online Microsoft community located here:

https://www.powershellgallery.com/

After installation try the following command to connect:

Connect-ExchangeOnline

Then I ran the following test command:

Get-EXOMailbox

Reference:
https://o365reports.com/2019/12/11/connect-exchange-online-powershell-without-basic-authentication/