--------------------------------------------------------------------------------------

How to COM-MIDL Custom Marshal Across Process Boundaries Function Calls that pass Pack(1) Structures containing pointers to Structures

---------------------------------

Note: This outline assumes that on your machine you have installed Windows 2000, Visual Studio 6 (with Service Pack 4) followed by an installation of Visual Studio .NET 7 Beta 1

---------------------------------

Walkthrough Summary

Using the ATL Wizard of Visual Studio 7 (VS7), ATL COM server DLL 'ATLProject1.dll' and its Proxy/Stub DLL 'ATLProject1PS.dll' are generated. For testing, a simple pointer-based hierarchy of structures is defined in 'TypesToMarshal.h', with parent type 'UnmTyp2_t'. Structure packing on 1-byte boundaries is enforced with a '#pragma pack(1)' statement in 'TypesToMarshal.h'. The header file is imported into 'ATLProject1.idl'. VS7's Class Wizard is used to generate a COM object 'AtlComObj0' exposing a Custom Interface. A method 'ReadWriteTestFunc0()' is added that receives a '[in, out]' pointer to the hierarchy's parent type UnmTyp2_t'. For testing we provide a body for 'CAtlComObj0::ReadWriteTestFunc0()'. Windows 2000 COM+ Services are used to render 'ATLProject1.dll' an out-of-process server. In a separate instance of VS7, a MFC dialog (the COM client) is generated and test code is provided for its OnClickedOK() event handler. Explicit steps required to debug out-of-process are shown.

1. Using Visual Studio 7 (.NET Beta 1) generate a new ATL Project (COM server).

1a. Somewhere on your hard drive create a new Folder (directory) that will contain your VS7 solutions, projects and source files. Name the folder UnManagedCode.

1b. Create a new VS7 solution: 'UnManagedCode\Solution1\Solution1.sln'.
In VS7 Left-click the 'File' menu item. Select 'New / Project'.

In the 'New Project' Dialog: Under 'Project Types:' select 'Visual Studio Solutions'. Under 'Templates:' select 'Blank Solution'. Using the 'Browse' button point the 'Location:' field to your 'UnManagedCode' folder. In the 'Name:' field enter "Solution1" (without quotes). Left-click 'OK'.

1c. Add a new ATL Project: 'UnManagedCode\ATLProject1\ATLProject1.vcproj'.

In VS7 Left-click the 'File' menu item. Select 'Add Project / New Project'.

In the 'Add New Project' Dialog: Under 'Project Types:' select 'Visual C++ Projects / Win32 Projects'. Under 'Templates:' select 'ATL Project'. Using the 'Browse' button point the 'Location:' field to your 'UnManagedCode' folder. In the 'Name:' field enter "ATLProject1" (without quotes). Left-click 'OK'.

In the 'ATL Project Wizard - ATLProject1' Dialog: Left-click 'Application Settings'. For 'Server Type:' select the 'Dynamic Link Library (DLL)' radio button. Leave all 'Additional options:' unchecked. Note: In particular, do NOT check 'Allow merging of proxy/stub code'. Left-click the 'Finish' button.

Comment: For a Unchecked 'Allow merging of proxy/stub code' check-box, VS7 Beta 1 generates a separate project 'ATLProject1PS.vcproj' with which to build the proxy/stub DLL. If 'Allow merging of proxy/stub code' is checked, the wizard relies on a makefile mechanism but fails to generate the required .mk makefile. If you want to 'Allow merging of proxy/stub code' you may do so by copying and modifying the .mk file generated by the VS6 'ATL Project Wizard'.

1d. Mark 'ATLProject1PS' as 'StartUp Project' and Build the Solution.

In VS7 'Solution Explorer' ('View menu item / Solution Explorer') right-click on the 'ATLProject1PS' item. Left-click 'Set as StartUp Project' from the drop-down menu.

Right-click the 'Solution 'Solution1' (2 projects)' item.

Click 'Configuration Manager...' from the drop-down menu. Make sure that a check mark is present in the 'Build' column for the 'ATLProject1PS' row. Click the 'Close' button.

Comment: The proxy/stub DLL is activated by the registration step (call to regsvr32.exe in the 'Post-Build Event' Tab of project ATLProject1PS properties. To deactivate the proxy/stub, unregister 'ATLProject1PS.dll' using 'regsvr32.exe /u'. After unregistering 'ATLProject1PS.dll' you may find it necessary to re-register 'ATLProject1.dll' (with regsvr32.exe) and possibly re-install 'ATLProject1.dll' into COM+ (see Step 5, below).

Build 'Solution1': Left-click the 'Build' menu item, select 'Build Solution'.

2. Define the structure types to be marshaled and import them into 'ATLProject1.idl'.

2a. Add a new header file 'TypesToMarshal.h' to 'ATLProject1'.
In 'Solution Explorer' right-click the 'ATLProject1' item, select 'Add / Add New Item ...'.

In the Add New Item - ATLProject1' dialog: For 'Categories:' select 'Visual C++ / C++'. For 'Templates:' click 'Header File (.h)'. In the 'Name:' field type "TypesToMarshal.h" (without quotes). Left-click the 'Open' button.

2b. Define structure types.
Paste the following code fragment into 'TypesToMarshal.h':

// ----- Paste to TypesToMarshal.h begin

// TypesToMarshal.h

#pragma once

#pragma pack(1) // we will marshal structures packed on 1 byte boundaries.

typedef struct tagUnmTyp1

{

char charMember1; // so that '#pragma pack(1)' affects structure ram layout

int intMember1;

BSTR bstrMember1; // just to add interest

}

UnmTyp1_t;

typedef struct tagUnmTyp2

{

char charMember2; // so that '#pragma pack(1)' affects structure ram layout

int intMember2;

BSTR bstrMember2; // just to add interest

UnmTyp1_t* punmtyp1Member2; // a pointer to struct tagUnmTyp1

}

UnmTyp2_t;

#pragma pack()

// ----- Paste to TypesToMarshal.h end

2c. Import 'TypesToMarshal.h' into 'ATLProject1.idl'.
In 'Solution Explorer' expand the 'ATLProject1' item by clicking its '+' icon. Also expand the 'Source Files' item. Double-click the 'ATLProject1.idl' item. In the 'ATLProject1.idl' source code window, immediately below the 'import "ocidl.idl";' line but above the listing of attributes for 'library ATLProject1Lib', paste the following code snippet:

// ----- Paste into ATLProject1.idl begin

import "TypesToMarshal.h";

// ----- Paste into ATLProject1.idl end

Build 'Solution1': Left-click the 'Build' menu item, select 'Build Solution'.

3. Add a new 'ATL Simple Object' (COM Server) to ATLProject1: 'AtlComObj0'.

3a. In 'Solution Explorer' right-click the 'ATLProject1' item. Select 'Add / Add Class...'.
In the 'Add Class - ATLProject1' dialog: For 'Categories:' select 'Visual C++ / ATL'. For 'Templates:' click 'ATL Simple Object'. Left-click the 'Open' button.
In the 'ATL Simple Object Wizard - ATLProject1' dialog: Select the 'Names' Tab. In the 'Short name:' field enter "AtlComObj0" (without quotes). Select the 'Attributes' Tab. For 'Interface:' click the 'Custom' radio button and leave the 'Automation compatible' box Un-checked. Click the 'Finish' button.

Build 'Solution1': Left-click the 'Build' menu item, select 'Build Solution'.

4. Add a new Method to the 'IAtlComObj0' COM server Interface.

4a. Open the 'Class View' window.
Left-click the 'View' menu item. Select 'Class View'.

4b. Add a new method.
In 'Class View' expand the 'ATLProject1' item by clicking its '+' icon. In the expanded list right-click the 'IAtlComObj0' item and select 'Add / Add Method...'.

In the 'Add Method Wizard - ATLProject1' dialog for 'Method Name:' enter "ReadWriteTestFunc0" (without quotes). To define a calling parameter to this function, check the 'in' and 'out' boxes under 'Param Attributes:', for 'Parameter Type:' enter "UnmTyp2_t*" (without quotes), for 'Parameter Name:' enter "pUnmTyp2Param" (without quotes). Click the 'Add' button. The 'Parameter List:' should now read "[in,out] UnmTyp2_t* pUnmTyp2Param". Click the 'Finish' button.

The Wizard will modify 'ATLProject1.idl', 'AtlComObj0.h' and 'AtlComObj0.cpp', amongst others.

Build 'Solution1'.

4c. Add code to CAtlComObj0::ReadWriteTestFunc0(UnmTyp2_t* pUnmTyp2Param){}.

In 'Solution Explorer' expand the 'ATLProject1' item by clicking its '+' icon. Expand the 'Source Files' item. Double-click the 'AtlComObj0.cpp' item. In the 'AtlComObj0.cpp ' source code window, find the definition of CAtlComObj0::ReadWriteTestFunc0(UnmTyp2_t* pUnmTyp2Param){}. Paste the following code block into ReadWriteTestFunc0() immediately below the comment '// TODO: Add your implementation code here' but above the 'return S_OK;' statement:

{ //----- Paste this block into ReadWriteTestFunc0() begin

if(NULL != pUnmTyp2Param)

{ // pUnmTyp2Param read check begin

// check that values received are same as sent by client

ATLASSERT(pUnmTyp2Param->charMember2 == '2');

ATLASSERT(pUnmTyp2Param->intMember2 == 222);

if(NULL != pUnmTyp2Param->bstrMember2)

{

ATLASSERT(0 == wcscmp(pUnmTyp2Param->bstrMember2, L"222"));

}

if(NULL != pUnmTyp2Param->punmtyp1Member2)

{

ATLASSERT(pUnmTyp2Param->punmtyp1Member2->charMember1 == '1');

ATLASSERT(pUnmTyp2Param->punmtyp1Member2->intMember1 == 111);

ATLASSERT(pUnmTyp2Param->punmtyp1Member2->bstrMember1 != NULL);

ATLASSERT(0 == wcscmp(pUnmTyp2Param->punmtyp1Member2->bstrMember1, L"111"));

}

} // pUnmTyp2Param read check end

if(NULL != pUnmTyp2Param)

{ // pUnmTyp2Param write begin

auto UnmTyp1_t* pUnmTyp1= (NULL != pUnmTyp2Param ?

pUnmTyp2Param->punmtyp1Member2 : NULL);

if(NULL != pUnmTyp1)

{

// for an additional check we free and reallocate

// all server-side pointers allocated by the midl marshaller

{ // proof of concept block begin

if(NULL != pUnmTyp1->bstrMember1)

{

::SysFreeString(pUnmTyp1->bstrMember1);

pUnmTyp1->bstrMember1 = NULL;

}

::CoTaskMemFree(pUnmTyp1);

pUnmTyp1 = (struct tagUnmTyp1*)

::CoTaskMemAlloc(sizeof(struct tagUnmTyp1));

ATLASSERT(NULL != pUnmTyp1);

::memset(pUnmTyp1, 0, sizeof(*pUnmTyp1));

pUnmTyp2Param->punmtyp1Member2 = pUnmTyp1;

} // proof of concept block end

pUnmTyp1->charMember1 = '5';

pUnmTyp1->intMember1 = 555;

auto BSTR* pbstr1= &pUnmTyp1->bstrMember1;

auto int boolVal= ::SysReAllocString(pbstr1, L"555555");

ATLASSERT(boolVal);

}

pUnmTyp2Param->charMember2 = '6';

pUnmTyp2Param->intMember2 = 666;

auto BSTR* pbstr2= &pUnmTyp2Param->bstrMember2;

auto int boolVal= ::SysReAllocString(pbstr2, L"666666");

ATLASSERT(boolVal);

} // pUnmTyp2Param1 write end

} //----- Paste this block into ReadWriteTestFunc0() end

Build 'Solution1': Left-click the 'Build' menu item, select 'Build Solution'.

5. Configure a COM+ Server Application Package 'SSMarshalTestPackage' to expose Interface 'IAtlComObj0' Out-Of-Process.

5a. Left-click your system's 'Start' menu. Select 'Settings / Control Panel / Administrative Tools / Component Services'.

In the 'Tree' Tab of 'Component Services' expand and select 'Computers / My Computer / COM+ Applications'. Right-click the 'COM+ Applications' item and select 'New / Application'. In the 'Welcome to the COM Application Install Wizard' dialog click the 'Next>' button. Click the 'Create an empty application' button. For 'Activation type' click the 'Server application' radio button, and in the 'Enter a name for the new application:' field enter "SSMarshalTestPackage" (without quotes). Click the 'Next>' button. In the 'Set Application Identity' page click the 'Interactive user -' radio button, click the 'Next>' button. Click the 'Finish' button.

5b. (Optional) In the 'Tree' Tab of 'Component Services', right-click item 'SSMarshalTestPackage' and select 'Properties'. In the 'Advanced' Tab of the 'SsMarshalTestPackage Properties' dialog, for 'Server Process Shutdown' click the 'Leave running when idle' radio button. Then click the 'OK' button.

Comment: This optional step will ensure that once started, 'dllhost.exe' for 'SSMarshalTestPackage' will remain up-and-running. If you need to shut it down, in the 'Tree' Tab of 'Component Services', right-click item 'SSMarshalTestPackage' and select 'Shut down'.

5c. In the 'Tree' Tab of 'Component Services' expand as needed and left-click the 'Components' item under package 'SSMarshalTestPackage'. Make sure the empty 'Components' right-pane is visible on your monitor screen and positioned to receive a drag-drop operation (next step).

5d. Point an instance of 'Windows Explorer' to the directory containing your COM server 'ATLProject1.dll'.

Left-click your system's 'Start' menu. Select 'Run...'. In the 'Run' dialog enter "explorer.exe" (without quotes) in the 'Open:' field. Click the 'OK' button.

In the 'Folders' pane of Explorer expand and locate your 'UnmanagedCode' directory for this Walkthrough. Left-click the 'Debug' item at '... / UnmanagedCode / ATLProject1 / Debug'. In the (right pane) file list locate item 'ATLProject1.dll'.

5e. Drag item 'ATLProject1.dll' from the file list of the Explorer dialog and drop it into the empty 'SSMarshalTestPackage / Components' pane in the 'Component Services dialog. The 'SSMarshalTestPackage / Components' (right) pane should now display an icon/item 'ATLProject1.AtlComObj0.1' to indicate successful COM+ registration.

6. Modify project 'ATLProject1PS' properties to enable debug of dllhost.exe 's execution of COM+ package 'SSMarshalTestPackage'.

6a. In the 'Tree' Tab of 'Component Services', right-click item 'SSMarshalTestPackage' and select 'Properties'. In the 'General' Tab of the 'SSMarshalTestPackage Properties' dialog, copy to the system Clipboard the GUID that shows after the 'Application ID:' label by right-clicking the GUID and selecting 'Copy' from the drop-down menu.

6b. In VS7 'Solution Explorer' right-click project item 'ATLProject1PS' (your StartUp project) and select 'Properties' from the drop-down menu.

In the 'ATLProject1PS Property Pages' dialog, expand the 'Configuration Properties' item and click the 'Debug' item under it.

For the 'Command' option enter "C:\winnt\system32\dllhost.exe" (without quotes) (if C: is not your Windows 2000 installation drive, correct with the proper drive letter). For 'Command Arguments' type "/ProcessID:" (without quotes) and in the same field, at the blinking caret immediately after the ending ':', right-click and select 'Paste' from the drop-down menu to retrieve your GUID. When done, the 'Command Arguments' option should read something like: "/ProcessID:{B02AA397-2FB3-4393-9B60-9E91E68F0895}" (but of course, with your GUID value).

Click the 'Apply' button, then click the 'OK' button.

7. Test your COM+ install of 'ATLProject1.dll' and your ability to debug dllhost.exe 's execution of package 'SSMarshalTestPackage'.

7a. Start Task Manager: 'taskmgr.exe'.

Left-click your system's 'Start' menu. Select 'Run...'. In the 'Run' dialog enter "taskmgr.exe" (without quotes) in the 'Open:' field. Click the 'OK' button.

When the 'Windows Task Manager' dialog comes up, click its 'Processes' Tab and then click the 'Image Name' column heading until processes are sorted ascending by name. Position the scroll-bar so that you can see all instances of dllhost.exe (count the number presently executing).

7b. In VS7 click the 'Debug' menu item, select 'Start'. At least on additional 'dllhost.exe' item should appear in the 'Windows Task Manager' dialog. In VS7 again click the 'Debug' menu item, select 'Stop Debugging'. One 'dllhost.exe' item should disappear from the 'Processes' list of the 'Windows Task Manager' dialog.

8. Generate a new VS7 COM client MFC Application.

8a. Start execution of a NEW instance of Visual Studio 7 (devenv.exe).

8b. Create a new VS7 solution: 'UnManagedCode\Solution2\Solution2.sln'.
In VS7 (new instance) Left-click the 'File' menu item. Select 'New / Project'.

8c. Add a new MFC Application: 'UnManagedCode\MFCProject1\MFCProject1.vcproj'.

In VS7 Left-click the 'File' menu item. Select 'Add Project / New Project'.

In the 'Add New Project' Dialog: Under 'Project Types:' select 'Visual C++ Projects / Win32 Projects'. Under 'Templates:' select 'MFC Application'. Using the 'Browse' button point the 'Location:' field to your 'UnManagedCode' folder. In the 'Name:' field enter "MFCProject1" (without quotes). Left-click 'OK'.

In the 'MFC Application Wizard - MFCProject1' dialog, left-click 'Application Type'. For 'Select application type:' click the 'Dialog based' radio button. Left-click the 'Finish' button.
Build 'Solution2': Left-click the 'Build' menu item, select 'Build Solution'.

9. Add a OnClickedOk() event handler to your dialog's code base.

9a. Make sure your 'Resource View' window is showing.
Click the 'View' menu item. Select 'Resource View'.

9b. Make sure your 'Properties Window' is showing.
Click the 'View' menu item. Select 'Properties Window'.

9c. Make sure your 'MFCProject1' dialog resource is open in your 'Resource View'.
In the 'Resource View - MFCProject1' window expand 'MFCProject1.rc / Dialog' and double-click the item 'IDD_MFCPROJECT1_DIALOG'.

9d. Add the OnClickedOk() event handler.
In the 'Dialog Editor' window (where MFCProject1.rc is opened), click the 'OK' button. Amongst the icons at the top of your 'Properties' window locate and click the 'ControlEvents' icon (appears as a 'lightning bolt'). From the list of events left-click on the 'BN_CLICKED' item, click its empty 'value' space to show a drop-down combo button, click the drop-down button and select '<Add> OnClickedOk'.
In 'Solution Explorer' expand the 'MFC Project1' item. Double-click item 'MFCProject1Dlg.cpp' to open the file. At the bottom of the file you will find the CMFCProject1Dlg::OnClickedOk() function.

10. Paste a body into function CMFCProject1Dlg::OnClickedOk().

10a. Copy and paste into CMFCProject1Dlg::OnClickedOk(), the following block of code:

{ // Paste this block into your 'OnClickedOk () --- begin

auto HRESULT hr= S_OK;

{

static bool bDidCallCoinitialize= false;

if(!bDidCallCoinitialize) {

hr = ::CoInitialize(NULL);

ASSERT(S_OK == hr);

bDidCallCoinitialize = true;

}

auto CComPtr<IAtlComObj0> ccomptrAtlComObj0;

try

{

hr = ccomptrAtlComObj0.CoCreateInstance(

__uuidof(AtlComObj0)

);

ASSERT(S_OK == hr);

auto struct tagUnmTyp2* pUnmTyp2Param= NULL;

{ // pUnmTyp2Param write begin

auto struct tagUnmTyp1* pUnmTyp1= (struct tagUnmTyp1*)

::CoTaskMemAlloc(sizeof(struct tagUnmTyp1));

ASSERT(NULL != pUnmTyp1);

(*pUnmTyp1).charMember1 = '1';

(*pUnmTyp1).intMember1 = 111;

auto BSTR bstr1= ::SysAllocString(L"111");

ASSERT(NULL != bstr1);

(*pUnmTyp1).bstrMember1 = bstr1;

auto struct tagUnmTyp2* pUnmTyp2= (struct tagUnmTyp2*)

::CoTaskMemAlloc(sizeof(struct tagUnmTyp2));

(*pUnmTyp2).charMember2 = '2';

(*pUnmTyp2).intMember2 = 222;

auto BSTR bstr2= ::SysAllocString(L"222");

ASSERT(NULL != bstr2);

(*pUnmTyp2).bstrMember2 = bstr2;

(*pUnmTyp2).punmtyp1Member2 = pUnmTyp1;

pUnmTyp2Param = pUnmTyp2;

} // pUnmTyp2Param write end

hr = ccomptrAtlComObj0->ReadWriteTestFunc0(

pUnmTyp2Param

);

ASSERT(S_OK == hr);

if(NULL != pUnmTyp2Param)

{ // pUnmTyp2Param read check begin

ASSERT(pUnmTyp2Param->charMember2 == '6');

ASSERT(pUnmTyp2Param->intMember2 == 666);

if(NULL != pUnmTyp2Param->bstrMember2)

{

ASSERT(0 == wcscmp(pUnmTyp2Param->bstrMember2, L"666666"));

}

if(NULL != pUnmTyp2Param->punmtyp1Member2)

{

ASSERT(pUnmTyp2Param->punmtyp1Member2->charMember1 == '5');

ASSERT(pUnmTyp2Param->punmtyp1Member2->intMember1 == 555);

ASSERT(pUnmTyp2Param->punmtyp1Member2->bstrMember1 != NULL);

ASSERT(0 == wcscmp(pUnmTyp2Param->punmtyp1Member2->bstrMember1, L"555555"));

}

// cleanup allocations made in this function

if(NULL != pUnmTyp2Param->punmtyp1Member2) {

if(NULL != pUnmTyp2Param->punmtyp1Member2->bstrMember1)

::SysFreeString(pUnmTyp2Param->punmtyp1Member2->bstrMember1);

::CoTaskMemFree(pUnmTyp2Param->punmtyp1Member2);

}

::CoTaskMemFree(pUnmTyp2Param);

} // pUnmTyp2Param read check end

}

catch(...)

{

ASSERT(FALSE);

}

} // Paste this block into your 'OnClickedOk () --- end

10b. At the top of file MFCProject1Dlg.cpp you will also need to paste the following snippet:

// paste to the include section of MFCProject1Dlg.cpp begin

#include <atlbase.h> // for CComPtr

#include "..\ATLProject1\TypesToMarshal.h" // required for #pragma pack(1)

#import "..\AtlProject1\Debug\AtlProject1.tlb" named_guids,\

no_namespace, exclude("tagUnmTyp2", "tagUnmTyp1");

// the 'exclude' clause is required for #pragma pack(1)

// paste to the include section of MFCProject1Dlg.cpp end

Comment: The above #import "AtlProject1.tlb" causes the pre-processor to create "AtlProject1.tlh" and #include it into 'MFCProject1Dlg.cpp'. You may open "AtlProject1.tlh" and verify that there, our user-defined types are declared as packed on 8-byte boundaries which is the rpc default but incorrect for us. The above 'exclude' clause is required.

Build 'Solution2': Left-click the 'Build' menu item, select 'Build Solution'.

11. Final Test.

11a. From your system's task bar select the instance of VS7 in which Solution1 (ATLProject1) is open. In 'Solution Explorer' expand item 'ATLProject1', double -click 'AtlComObj0.cpp' to open the file.
Place a 'Debugger Break Point' at entry to function CAtlComObj0::ReadWriteTestFunc0() by positioning the cursor on the first curly bracket '{' in the function and pressing 'function key' <F9>.

11b. From your system's task bar select the instance of VS7 in which Solution2 (MFCProject1) is open. In 'Solution Explorer' expand item 'MFCProject1', double -click 'MFCProject1Dlg.cpp' to open the file.
Place a 'Debugger Break Point' on the line of code at which 'ReadWriteTestFunc0()' is called. Also place a break point on the ASSERT statement immediately after that call.

11c. Returning to VS7 for ATLProject1, start the debugger by pressing function key <F5>. A new dllhost.exe item should show in your 'Windows Task Manager' Processes list.

11d. In VS7 for 'MFCProject1' press <F5> to begin execution. Click the 'OK' button and, at the breakpoints, to step through the code press <F10>.

Done.

Rafael Pena

rpenaphd@worldnet.att.net

3/5/2001

Keywords: HowTo