Visual Studio 2019 C++/CLI Building without MSBuild

Hello, I am trying to port a C++/CLI project to target .NET Core without MSBuild and I keep getting the following errors:

Error 1:
c:\dotnet\sdk\3.1.105\microsoft\microsoft.net.build.extensions\net461\lib\system.runtime.dll: fatal error C1001: An internal error has occurred in the compiler.
(compiler file 'msc1.cpp', line 1528)
 To work around this problem, try simplifying or changing the program near the locations listed above.
Please choose the Technical Support command on the Visual C++
 Help menu, or open the Technical Support help file for more information
Internal Compiler Error in C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.24.28314\bin\HostX64\x64\cl.exe.  You will be prompted to send an error report to Microsoft later.
INTERNAL COMPILER ERROR in 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.24.28314\bin\HostX64\x64\cl.exe'

Error 2:
c:\dotnet\sdk\3.1.105\microsoft\microsoft.net.build.extensions\net461\lib\system.runtime.dll: fatal error C1001: An internal error has occurred in the compiler.
(compiler file 'msc1.cpp', line 1528)
 To work around this problem, try simplifying or changing the program near the locations listed above.
Please choose the Technical Support command on the Visual C++
"(I believe this is a parse error, found something saying this in microsoft support pages)"
Help menu, or open the Technical Support help file for more information

When I researched these errors I found that most people rebuilt the project and it went away,
as I am not using MSBuild this is not an option.  To build I am using a Makefile with /clr:netcore,
this builds if I change the Makefile to target .NET Framework.

I have tried the following:
Deleting the old dll and rebuilding
Including mscrolib, and system.runtime using /FU
Including the directories using /AI
Linking in the ijwhost.lib using LINKLIBS, CPPFLAGS, and LIBDEPS
Creating a runtime config file in the following format:

{
  "runtimeOptions": {
    "tfm": "netcoreapp3.1",
    "framework": {
      "name": "Microsoft.NETCore.App",
      "version": "3.1.0"
    }
  }
}

Moving the runtime config file to the output directory
Removing Optimiazations
Adding ijwhost.dll to the output directory

I am using vs2019 (version 16.4.10) with the following attributes:
Windows 10 SDK (10.0.17134.0)
MSVC v142 - VS 2019 C++ x64/x86 build tools (v14.24)
C++/CLI support for v142 build tools (14.24)
And the following .NET Core runtimes:

.NET Core 2.1 LTS Runtime

.NET Core 2.2 Runtime (EOL)

.NET Core 3.0 Runtime

.NET Core 3.1 SDK