Problem:

Imagine that you've got an AddressBook COM server running on some machine on the network (or just in some other apartment). It exposes one main Interface - IAddressBook, that allows you to find, create, and modify "Entries".

 
	interface IAddressBook : IUnknown
	{
		[helpstring("method FindEntry")] HRESULT FindEntry([in] BSTR name, [out, retval] IEntry **ppEntry);
		[helpstring("method AddEntry")] HRESULT AddEntry([in] BSTR name, [out, retval] IEntry **ppEntry);
	};
 

IEntry is basically a collection of properties related to a particular address:

interface IEntry : IUnknown
{
	[propget, helpstring("property StreetNumber")] HRESULT StreetNumber([out, retval] short *pVal);
	[propput, helpstring("property StreetNumber")] HRESULT StreetNumber([in] short newVal);
	[propget, helpstring("property StreetName")] HRESULT StreetName([out, retval] BSTR *pVal);
	[propput, helpstring("property StreetName")] HRESULT StreetName([in] BSTR newVal);
	[propget, helpstring("property ZipCode")] HRESULT ZipCode([out, retval] long *pVal);
	[propput, helpstring("property ZipCode")] HRESULT ZipCode([in] long newVal);
	[helpstring("method Commit")] HRESULT Commit();
};

People connect to the AddressBook server by calling CoCreateInstance, then they get and set the various properties. When they are done making changes and want the changes to be reflected, they call "Commit()".

Other examples of this sort of scenario are:

• You have some class that creates "Request" interfaces that can be manipulated by setting a series of properties (e.g. buy/sell, number of lots etc.). Then you "Send()" the request to get back a "Response". The response is just another interface with a whole bunch of properties that you want to read to get the answer.
• You have a theoretical model which you set up by a collection of properties (bid price, ask price, etc.) Then you call "Calculate()", which returns an interface with all of the results (delta, gamma, theoretical value etc.).

Anyway, my point is, it's a general problem.

OK, so we have this interface, and we might write a client that looks like this:

 
#include <windows.h>
#include <iostream>
#import "..\AddressBook\debug\AddressBook.dll"
 
int main() {
 CoInitialize(0);
 
 {
  using namespace ADDRESSBOOKLib;
  using namespace std;
 
  IAddressBookPtr addressbook;
 
  
  HRESULT hr = ::CoCreateInstance(__uuidof(AddressBook),
         0,
         CLSCTX_ALL,
         __uuidof(IAddressBook),
         reinterpret_cast(&addressbook));
  if(FAILED(hr))
  {
   _com_error er(hr);
 
   cerr << "COM ERROR: " << er.ErrorMessage() << endl;
   CoUninitialize();
   return 0;
  }
 
  try {
   IEntryPtr entry = addressbook->AddEntry("Paul");
   entry->StreetName = "Racine";
   entry->StreetNumber = 2021;
   entry->ZipCode = 60614;
 
   entry->Commit();
 

   cout << "Paul: " << entry->StreetNumber <<
    " " << (const char *)(entry->StreetName) << " " <<
    entry->ZipCode << endl;
 
   IEntryPtr found_entry = addressbook->FindEntry("Paul");
   cout << "Paul retrieved from Server: " << found_entry->StreetNumber <<
    " " << (const char *)(found_entry->StreetName) << " " <<
    found_entry->ZipCode << endl;
 
  } catch (const _com_error &e) {
   cerr << "COM ERROR: " << e.ErrorMessage() << endl;
  }
 }
 
 CoUninitialize();
 return 0;
}

This sample client just creates an Entry for "Paul" and sets the address. Then, just to test that it worked, I "Find" the entry again and display it.

Here is the output:

Paul: 2024 Halsted 60657
Paul retrieved from Server: 2024 Halsted 60657

Here is the crux of the problem: If my AddressBook Server happened to be in Frankfurt (and the client is in Chicago), then each time I get or set the property, it results in a costly round-trip over the network.This is because my "Entry" exists on the server in Frankfurt. When the Client in Chicago calls "AddEntry", the method call is remoted to the Frankfurt server, which creates an internal IEntry implementation. It then "marshals" this interface and returns this to the client in Chicago.

COM, behind the scenes, creates a "Stub" on the machine in Frankfurt, which converts RPC method calls into local method calls on my IEntry implementation. The Unmarshalled interface on the client in Chicago results in COM creating a "Proxy" that implements "IEntry", by converting calls made by the client into RPC method calls that connect back to the Stub in Frankfurt. In COM, this is called "COM Standard Marshalling", because it's what you get by default when you pass an Interface between apartments.

Each time I set a property on the client, it goes through the COM supplied "Proxy" which converts it into an RPC call. This RPC call goes over the network and is invoked on the Stub in Frankfurt, which converts it back into a method call that is made on the server IEntry implementation. Needless to say, this is extremely inefficient, especially if we have a lot of properties to set. Each "getting" or "setting" of a property results in a round trip over the network!

What is needed here is a more efficient interface. We need an IEntry interface that allows the client to group together all of the required property changes into a single method call that results in only one round trip. Something like this:

 
interface IEntryStream : IUnknown
{
	[helpstring("method Update")] HRESULT Update([in] BSTR propertystring);
};

We come up with some convention, or "record format", (e.g. each property is delimited by spaces), which is encoded by the client and decoded by the server. The client would make changes like this:

 
	IEntryStreamPtr entry = addressbook->AddEntry("Paul");
	entry->Update("2021 Racine 60614");

The downside to this is that we have now jumped back 20 years back into the past - alarm bells should have started ringing as soon as I said "Record format"!. All of the problems that COM was meant to solve come back - if we want to add a new property, we have to fit it into our record format, but more importantly, every Client now needs to know about this new property in order to correctly read the record. It's ugly because the client ends up writing code to "encode" information into the Record format, which always ends up being messy.

The code for parsing and unparsing the record, which should be part of the "Server", ends up being duplicated over and over in each of the different clients.

But there is no denying that this is the most efficient way of doing things when a network is involved.

Consider also the case when the Server is Inproc. Using IEntryStream is actually less efficient than IEntry because the client and server end up needlessly packaging and unpackaging this data as records.

Fortunately, with Custom marshalling, COM gives us a way to have our cake and eat it too. Ideally, what we want is the following to happen:

When the IEntry implemention on the Server is passed back to the client, a local "inproc" version of IEntry is constructed in the client's apartment that has a copy of the properties contained in the server. Then, when the client gets or sets properties, its local version of the IEntry interface just remembers the changes, but doesn't actually communicate back to the server. Then, when the client calls "Commit()", its local IEntry implementation connects back to the Server's "IEntryStream" interface and calls "Update", once. Furthermore, if the Server is actually INPROC with the client, we don't want to bother with any of this at all - we just return the actual IEntry interface directly to the client.

COM Custom marshalling allows us to implement this solution, in a way that is completely invisible to the client. The client has no idea that this is how we are performing its changes, except that it will be much faster!

The way that we indicate to COM that we wish to use Custom marshalling is by making our instance of whatever object we want to marshal implement the "IMarshal" interface.

[
    local,
    object,
    uuid(00000003-0000-0000-C000-000000000046)
]
 
interface IMarshal : IUnknown
{
 
    typedef [unique] IMarshal *LPMARSHAL;
 
    // GetUnmarshalClass
    // Implemented by Server (Stub) side only -
    // Return CLSID of the custom Proxy that we want instantiated on the client

    HRESULT GetUnmarshalClass
    (
        [in] REFIID riid,
        [in, unique] void *pv,
        [in] DWORD dwDestContext,
        [in, unique] void *pvDestContext,
        [in] DWORD mshlflags,
        [out] CLSID *pCid
    );

    // GetMarshalSizeMax
    // Implemented by Server side only -
    // Return maximum number of bytes that our "marshalled" form should take

    HRESULT GetMarshalSizeMax
    (
        [in] REFIID riid,
        [in, unique] void *pv,
        [in] DWORD dwDestContext,
        [in, unique] void *pvDestContext,
        [in] DWORD mshlflags,
        [out] DWORD *pSize
    );

    // MarshalInterface
    // Implemented by Server side only -
    // Serialize all information required to initialize the proxy on the Client
 
    HRESULT MarshalInterface
    (
        [in, unique] IStream *pStm,
        [in] REFIID riid,
        [in, unique] void *pv,
        [in] DWORD dwDestContext,
        [in, unique] void *pvDestContext,
        [in] DWORD mshlflags
    );

    // UnmarshalInterface
    // Implemented by Client (Proxy) side only -
    // read the stream of bytes and put
    // the specified interface into *ppv

    HRESULT UnmarshalInterface
    (
        [in, unique] IStream *pStm,
        [in] REFIID riid,
        [out] void **ppv
    );

    // ReleaseMarshalData
    // Needs to be implemented by both
    // Custom Proxy and the Server side.
    // Free any resources that might be
    // "allocated" in the stream (e.g.
    // if another COM interface is serialised
    // in the stream, you should call CoReleaseMarshalData
    // on that part).

    HRESULT ReleaseMarshalData
    (
        [in, unique] IStream *pStm
    );
 
    // DisconnectObject is only called if the server in Frankfurt calls
    // "CoDisconnectObject". We call CoDisconnectObject when we want to
    // rudely boot out any clients of this COM server and shutdown immediately.
    // You can leave it unimplemented if your server side code never calls
    // CoDisconnectObject.

    HRESULT DisconnectObject
    (
        [in] DWORD dwReserved
    );
}

Basically, in COM, references to interfaces can always be "serialized" into a stream of bytes that can be sent almost anywhere, or even saved to a file (but only for about 6 minutes before DCOM will garbage collect it away). Another word for "Serializing" (which is more of an OO term relating to persisting the actual state of an object rather than just a "reference" to it) is "Marshalling" which is the term used by COM (which I guess it got from RPC).

Whenever you have a COM interface that you want to be able to use somewhere else, you have two choices:

1. Pass it as a parameter in a method call on a COM interface (i.e. the normal way!)
2. Serialize it using "CoMarshalInterface" into a stream of bytes. Send the stream of bytes where you want it. Then call CoUnmarshalInterface on the other end.

Method 2 is exactly what your MIDL generated proxy stub code does when you use method 1. The ProxyStub marshalling code generated by MIDL (and the oleautomation marshaler etc.) simply generates calls to CoMarshalInterface/CoUnmarshalInterface on the stub and proxy respectively.

So here is the sequence of events as they occur now with the Custom marshalled IEntry implemention:

• Client in Chicago calls AddEntry, and the AddressBook Server in Frankfurt creates an internal implementation of IEntry, that just happens to also implement IMarshal.
• The IAddressBook proxy/stub code generates a call to "CoMarshalInterface" on the server in Frankfurt, passing in the interface pointer, the interface ID, an IStream pointer, and the context (whether its to another thread in the current process, another process on the same machine, or another machine on the network).
• Inside CoMarshalInterface: CoMarshalInterface QI's the IEntry implementation (i.e. calls QueryInterface()) for IMarshal, which this time doesn't fail with E_NOINTERFACE, but rather returns an interface pointer that we'll call pM.
• It then calls pM->GetUnmarshalClass to find out the CLSID of the custom proxy that it should instantiate using CoCreateInstance on the client in Chicago.
• It then calls pM->GetMarshalSizeMax to find out how many bytes to allocate to be sent back to Chicago.
• It then calls pM->MarshalInterface. This tells the IEntry implemention to write out a stream of bytes that can be used to initialize the Custom Proxy in Chicago. We can write whatever we want into this stream of bytes. In our case, we write out two things:
  1. A CoMarshalled IEntryStream interface (which ends up being marshalled using standard marshalling)
  2. A copy of the properties i.e. the StreetNumber, StreetName and ZipCode
  #1 is the recursive trick by which we communicate back to the server from our custom client. Note that the Call to CoMarshalInterface actually generates a recursive call to our IMarshal implementation again! However, the IID passed in the second time around will be different which allows us to differentiate and forward to the COM Standard Marshaller (see below.)

  This call to the Standard Marshaller to marshal the IEntryStream interface results in a Stub generated for our IEntryStream implementation.

• CoMarshalInterface prefixes our stream of bytes with a flag indicating that custom marshalling was used, and the CLSID that we returned from GetUnmarshalClass.
• So our marshalled interface pointer stream of bytes looks sort of like this (assuming our Entry originally has the address 2156 Racine 60614)

  		**********************
  		flag indicating custom marshalling
  		**********************
  		CLSID of Proxy to instantiate with CoCreateInstance(Inproc)
  		***********************
  		Marshalled IEntryStream interface
  		***********************
  		"2156 Racine 60614"
  	

• This resultant stream of bytes is returned by CoMarshalInterface to the IAddressBook Stub code, which sends it to the IAddressBook proxy in Chicago.
• On the machine in Chicago, the IAddressBook proxy stub code generates a call to CoUnmarshalInterface with the interface pointer that we returned.
• CoUnmarshalInterface looks at the input stream and sees that Custom marshalling was used. It then reads the CLSID and instantiates an Inproc instance of our CLSID.
• It then QI's this inproc instance for IMarshal, which returns our custom Marshalling interface for our Proxy. Our inproc proxy implementation implements IMarshal and returns this interface into itself. We'll call this pM again.
• CoUnmarshalInterface then calls pM->UnmarshalInterface, with the remainder of the stream of bytes that we wrote out in our MarshalInterface method on our server.
• In this case, we implement our UnmarshalInterface method on the custom Proxy by first calling CoUnmarshalInterface to get back the IEntryStream pointer. This act ends up instantiating a COM standard Proxy which communicates via RPC with the Stub that was instantiated in Frankfurt.
• Then we deserialize the properties by reading the rest of the properties into the member variables of the custom Proxy
• This pointer to our custom Proxy is returned to the client.

The client can get and set the properties, which our custom Proxy implements simply by getting and setting an in memory copy. When the client calls Commit(), we collect together all of the properties (the street name, street number and zip code) and call IEntryStream->Update to propagate them back to the server.

A name for this technique (marshalling an interface as part of another interface) that I came across reading some web pages is "Custom Handling".

Note finally, that if the server is Inproc with the client, the original IEntry interface is returned directly to the client. None of the proxy stub code gets executed in this case, not even the QI for IMarshal.

The Standard Marshaller.

The main problem with supporting IMarshal for an interface is that all interfaces exposed by that object through QI must be supported. Once we've said we know how to custom marshal an object, we have to support marshalling all interface pointers that that object implements. Calling for these cases CoMarshalInterface is not good enough because, as I said before, it simply QI's us again for IMarshal and would end up in an infinite recursive loop.

However, what we can do is get an instance of the COM standard marshaller (which generates a normal proxy and stub between the client and server) by calling a COM API function called CoGetStandardMarshal. This resultant interface does not call back on us.

Whenever we therefore end up having to support IMarshal for an interface that we don't support, we can simply forward to the IMarshal supplied by CoGetStandardMarshal. In the case of IEntryStream, that's what I end up doing.

Conclusion:

Using COM Custom Marshalling allows you to resolve the implicit tension between writing tight, clean interfaces with a lot of semantic content, and writing network-efficient interfaces that minimise round-trips.