IIS ASP and the Shell CScript/WScript hosts are just environments for the COM-based Active Scripting engine.
At the time it was very capable as lots of software titles opened themselves up to automation via COM - that's why you could have Classic ASP pages that invoked Office Excel to generate charts dynamically.
Unfortunately it was all a mess, but it was also something beautiful because it meant that a desktop/server OS was more than just a place where heterogenous applications could live: they could truly interact with each other regardless of their programming language or runtime platform.
That grand vision - itself an extension of OLE from the early 1990s - died down with the rise of .NET (this wasn't intentional - Microsoft just had a lot of reorgs around that time and COM was more-or-less feature-complete, and no-one was really using DCOM) - but with Windows 8 and the "WinRT" API I was hopeful that COM would be back and new ways of automating applications would come around, and that Microsoft would have a great system for non-programmer users wanting to build workflows like they can with Apple's macOS Automator and AppleScript.
This also ties-in with the do-as-little-work-as-necessary being done for VBA support in Microsoft's products. While we all wish Microsoft would allow modern JavaScript as a first-class COM automation language - the Office VBA runtime only supports its own VB6-dialect (the VBA language), and while Active Scripting supports JScript (a snapshot of JavaScript version 3 from 1999-2001) it doesn't support static typing nor is any of the tooling inside Office compatible at all. - so instead we have JavaScript, but only for "Office Apps" which don't offer anywhere near the same kind of integration or control over Office applications (or things like filesystem access) that VBA does. It's like it's 2008 again and everyone's excited about the new iPhone OS App Store and thinking about all the cool iPhone OS system extensions they'd add, like a Today screen or a camera app that can record video by using low-level camera hardware APIs - only to be discover Apple will only let-in apps using nerfed and siloed high-level APIs that take away all of the fun from extending an OS.
I wouldn't say that COM died with .NET - .NET had COM support baked deeply into the platform, to the point of language hacks where to this day, you can "new" a specially defined interface in C# (originally it was a way to reference the associated default coclass for a COM interface, if any). It's very easy to use and create COM libraries, and WinForms fully supports ActiveX controls.
Even as late as 2010, COM support in .NET was still being developed - C# 4 got named indexers for this reason alone, and its "dynamic" supported IDispatch out of the box - its main scenario being Office automation.
I argue that COM "died" when .NET rose in reference to the COM ecosystem because prior to the release of .NET on Windows most non-Java developers were writing in either VB6/Delphi or C/C++ for their desktop applications.
VB6 and C++ (specifically, Microsoft VisualC++'s __declspec-punctuated dialect at least) were both intimately connected to COM with their respective compilers have built-in support for DOM/DCOM/etc (e.g. VC processing IDL files and generating TLB files, while the VB6 compiler seemingly reads-in TLB files natively - and can generate its own too) - and all major components for Windows desktop development (such as ADODB for OLE and ODBC database access, the Macromedia Flash player, OLE objects in general, and VB-ecosystem "components" like charting widgets and new styles of buttons were all done using COM - which means programs written in VB6 running in the VB runtime could use the same GUI components and platform feature libraries as C/C++ code running in its own world - that is cool!
While .NET is fully compatible with COM - the ecosystem isn't the same: WinForms and WPF GUI components written for .NET now cannot be used by VB6 forms or C/C++ windows (at least not without hosting .NET in-process and then facing additional issues integrating your Win32/User32 GUI with a WinForms hosting surface) - while hosting COM GUI components in WinForms is straightforward and just like VB6, hosting a COM GUI component in a WPF environment is very, very painful (google "wpf airspace" to see why this is a bad problem).
Additionally, consuming COM from .NET (even VB.NET) isn't as easy or as straightforward as it is with VB6 either - as you have to contend with the ISA of your .NET process (e.g. you can't load 32-bit COM components into an x64 .NET process).
I feel the amount of third-party software on Windows that offered COM automation has dwindled in recent years because of those reasons.
I think we need to distinguish between COM and OLE/ActiveX here. COM interop in .NET is two-way, and the consumer doesn't even need to be aware that there's a VM running the code that they're invoking. OLE is much more challenging, because the UI framework needs to be aware of it, and because it's built on Win32 UI primitives - so any framework that doesn't use those is going to have issues much like WPF (Qt is another example).
Bitness is not a new problem - VB6 similarly has problem consuming COM components written in C++, but compiled as 64-bit. Or, to put it another way: VB6 didn't have a problem, because 64-bit wasn't a thing back when it was popular, and didn't really became common on Win32 until relatively recently. But in a similar vein, if you always compile .NET code targeting x86 rather than AnyCPU (which is the default for new .NET VS projects - not sure when that changed, but it's been several years), you get the same exact behavior as VB6.
Getting back to OLE and ActiveX - I would say that it was killed by Microsoft directly, not because of .NET. Back in the day, most non-trivial Windows apps made by MS supported OLE for embedding - this was most commonly used with IE, but you could also do it with e.g. Word and Excel... until you couldn't anymore. If I remember correctly, it was Office 2007 that killed it as an officially supported scenario (you could still do it, but if anything broke, you were on your own). That roughly coincides with WPF, and I don't think that's a coincidence - but it doesn't mean that WPF/.NET was the primary cause.
> COM interop in .NET is two-way, and the consumer doesn't even need to be aware that there's a VM running the code that they're invoking.
I understand that's because .NET components loaded into a native process via COM are done so in-proc but using a single instance of the CLR, means you can't have .NET 2.0 CLR and .NET 4.0 CLR components loaded at the same time into the same process. I also understand (but am unsure) about what this means for AppDomains.
.NET Core simplifies things considerably: by being utterly broken when it comes to COM support and generally requiring each .NET Core program to have its own independent process rather than supporting multiple AppDomains in the same process (okay, that's a bit of hyperbole: .NET Core supports strongly-typed COM objects but they removed support for using `dynamic` with late-bound COM objects - and I can't find any documentation regarding loading .NET Core components via COM into a native VB6/C++ process.
> But in a similar vein, if you always compile .NET code targeting x86 rather than AnyCPU (which is the default for new .NET VS projects - not sure when that changed, but it's been several years), you get the same exact behavior as VB6.
I understand the default for .NET Framework application projects is still AnyCPU but a new option called "Prefer 32-bit" is selected by default. Whereas for library projects and all .NET Core projects the default is AnyCPU ("Prefer 32-bit" is only an option for *.exe projects).
> If I remember correctly, it was Office 2007 that killed it as an officially supported scenario (you could still do it, but if anything broke, you were on your own).
> That roughly coincides with WPF, and I don't think that's a coincidence - but it doesn't mean that WPF/.NET was the primary cause.
I understand the reason was because very few applications in the wild depended on this functionality (that's what Microsoft's telemetry in Windows is for) - I think the only real application that made full use of this was MS Binder (rip 2002) - and I only ever got annoyed at the times when Internet Explorer would load Office Word or Excel directly into the browser shell.
Office 2007's radically redesigned UI required much more screen real-estate - whereas Office's support for being loaded into an OLE window was back when people were running at 640x480 or 800x600 (if you were lucky!) and Office had 16px-sized toolbar buttons - which is another reason to avoid wasting developer time working on it.
> I understand that's because .NET components loaded into a native process via COM are done so in-proc
That's not necessarily so. By default, a .NET class exposed via COM is free-threaded, so it'll be instantiated in-proc. But you can have all the same complicated apartment arrangements as with native - and marshaling is done via type libraries that are generated from assembly metadata. So if you want an out-of-proc COM server written in .NET, it's quite possible.
> but using a single instance of the CLR, means you can't have .NET 2.0 CLR and .NET 4.0 CLR components loaded at the same time into the same process.
If you use in-proc components, then yes, the runtime is loaded. But .NET 2 and .NET 4 runtimes are intentionally designed to be possible to side-load. They don't see each other as .NET, of course - so if you take a reference to an object in one runtime, and pass it through native code as a COM reference to the other runtime, it'll see it as an opaque COM component.
.NET Core is its own thing - it's clear that everything Win32 is legacy there, including COM. I was surprised it even got WinForms...
The new ways of automating applications is Powrshel with its COM/.NET/DLL integration.
And UWP as modern COM is here to stay, no matter what, Microsoft seems to be willing to spend the money to keep it going, even it requires a new programmer generation.
At the time it was very capable as lots of software titles opened themselves up to automation via COM - that's why you could have Classic ASP pages that invoked Office Excel to generate charts dynamically.
Unfortunately it was all a mess, but it was also something beautiful because it meant that a desktop/server OS was more than just a place where heterogenous applications could live: they could truly interact with each other regardless of their programming language or runtime platform.
That grand vision - itself an extension of OLE from the early 1990s - died down with the rise of .NET (this wasn't intentional - Microsoft just had a lot of reorgs around that time and COM was more-or-less feature-complete, and no-one was really using DCOM) - but with Windows 8 and the "WinRT" API I was hopeful that COM would be back and new ways of automating applications would come around, and that Microsoft would have a great system for non-programmer users wanting to build workflows like they can with Apple's macOS Automator and AppleScript.
This also ties-in with the do-as-little-work-as-necessary being done for VBA support in Microsoft's products. While we all wish Microsoft would allow modern JavaScript as a first-class COM automation language - the Office VBA runtime only supports its own VB6-dialect (the VBA language), and while Active Scripting supports JScript (a snapshot of JavaScript version 3 from 1999-2001) it doesn't support static typing nor is any of the tooling inside Office compatible at all. - so instead we have JavaScript, but only for "Office Apps" which don't offer anywhere near the same kind of integration or control over Office applications (or things like filesystem access) that VBA does. It's like it's 2008 again and everyone's excited about the new iPhone OS App Store and thinking about all the cool iPhone OS system extensions they'd add, like a Today screen or a camera app that can record video by using low-level camera hardware APIs - only to be discover Apple will only let-in apps using nerfed and siloed high-level APIs that take away all of the fun from extending an OS.