To query any SQL database from PowerShell, you need to have the SQLPS module installed along with a few supporting packages. I am going to be focusing on the SQL Server 2012 packages in this article.

Technical documentation for Microsoft SQL Server, tools such as SQL Server Management Studio (SSMS) , SQL Server Data Tools (SSDT) etc. - MicrosoftDocs/sql-docs ...

While the below diagram is my original goal, it turns out Azure Relay passes my client machine's IP address to SQL Database. This defeats the point of the relay (from a firewall rule persepective).