I'm working on a debugger using the ICorDebug interfaces. I'vem only just started and I'm trying to get a proof of concept for my boss. I'm able to launch or attach to managed apps and I'm getting all the proper DebugManagedCallback calls for attach, create process, load module, etc.

Which thread in the debugger is the one calling Continue? ICorDebug does have a rather frustrating restriction that you can'tcall Continue(OutOfBand=false) on the thread that dispatches the ICorDebugUnmanaged::DebugEvent callback. This looks like the issue you're hitting above. In .Net 2.0, I'd expect you get a CORDBG_E_CANT_CALL_ON_THIS_THREAD failure on Continue if you tried.

Random trivia re your handler:
- It's dangerous to clear all exception events because that will interefer with exceptions that the debuggee expects to have thrown. A simple adjustment would be to check the exception code and only clear Breakpoints.
- I think your LoadDll needs to close a file handle in the debug event.

FWIW, the latest MDbg sample has managed C# wrappers for the native debugging API, which you may find useful.

"I'm using the CLR version returned from GetRequestedRuntime() for creating a process and GetVersionFromProcess() for DebugActiveProcess()"
What exact version string is that? "v1.0.3705"? "v1.1.4322"? "v2.0.50727"?

Not only that you can, but you have to. Asa very quick prototype, you can have your Continue just spin up a new thread each time, and that threadProc just calls ICorDebugProcess.Continue(oob=false). Obviously, having a new thread each time is not efficient, but it will let you quickly check whether this is the problem.

For a managed-only debugger, you should definitely check out the Mdbg sample regarding consuming ICorDebug.

Writing an interop-debugger is very very complicated, and unfortunately, we don't have it well documented. This will be a very bumpy process. If a managed-only debugger is sufficient for your needs, you'll probably want to stick with that.

In this case, I'm guessing that you're not clearing the exception (ICDProcess::ClearCurrentException) and so the int3 is going unhandled and tearing down the process.

Thanks for the quick answer, Mike. I don't know if I would have even gotten this far without your blog.

I'm not sure what problem you're hitting with the initial int3. After the Clear + Continue, what happens?

Note that there are several different things here:
1. Unmanaged breakpoints: you really do need unmanaged debugging for this.
2. viewing unmanaged threads and modules: you can actually do this from managed-only debugging by using non-invasive inspection. For example, kernel32!CreateToolHelp32Snapshot should still work when you're managed-only debugging.
3. Get a mixed callstack: you can actually do this from managed-only debugging too! If unmanaged code calls managed code; you can still get the native frames.

You may be able to get what you want without having to go through the pain of implementing an interop-debugger?

I'm running my debugger in Visual Studio 8. I have a breakpoint set on EXCEPTION_DEBUG_EVENT of the unmanaged callback function as well as breakpoints set on the create thread and load module managed callback functions.

I hit the breakpoint in the unmanaged callback. I call ClearCurrentException() which returns S_OK. I then call Continue() and finally press F5 for Go in VS. The managed callback breakpoints are never hit and I never see the app i'm debugging on the screen. I can see it in the process list of Task Manager but not the application list. So, I'm assuming it's hung somewhere.

RE interop debugging:: You can debug a C# app in VS and trace into an unmanaged C++ dll. I need to be able to do that too.

Thanks, as always in advance. The docs on ICorDebug are so few and the docs posted on MSDN are already out of date. Without your blog and help, I don't know how I'd do it.

Sorry for my delay in answering. My response is slower because I don't know the answer here . I can't piece things together from your description. There's too many potential questions such as: is this attach or launch? are you getting an out-of-band events? what exceptions are you getting (just the startup int3 or anything else)? What other debug events are you getting? What OS are you running on?

I'm thinking the best approach here may be to see if I can provide a small MDbg extension that uses interop-debugging to print native debug events. Although that wouldn't bea full interop-debugger, it would show how to use the basic functionality. You could then step through that as a working example which would hopefully answer lots of questions.

"You can debug a C# app in VS and trace into an unmanaged C++ dll. I need to be able to do that too."
Yeah, that requires interop-debugging.

"docs posted on MSDN are already out of date"
True for http://msdn.microsoft.com. The dos on MSDN2 (ICorDebug at http://msdn2.microsoft.com/en-us/library/ms230588.aspx ) should be up to date, althoughstill sparse.

What thread are you calling ICorDebugProcess::Continue on?
Which version of the CLR is running in the debuggee?

>> What thread are you calling ICorDebugProcess::Continue on?

All ICorDebugProcess::Continue() takes is a BOOL for out of band. The ICorDebugProcess interface I'm using is the one returned from DebugActiveProcess() or AttachToProcess()

Here's my implementation of DebugUnmanagedCallback::DebugEvent()

HRESULT STDMETHODCALLTYPE CDebugUnmanagedCallback::DebugEvent(LPDEBUG_EVENT pDebugEvent, BOOL fOutOfBand)

{

switch (pDebugEvent->dwDebugEventCode)

{

    case CREATE_PROCESS_DEBUG_EVENT:

        CloseHandle(pDebugEvent->u.CreateProcessInfo.hFile);

    break;

    case EXCEPTION_DEBUG_EVENT:

        m_pSymInterface->ClearCurrentException(pDebugEvent->dwThreadId);

        break;

    case LOAD_DLL_DEBUG_EVENT:

    break;

    case UNLOAD_DLL_DEBUG_EVENT:

    break;

    case CREATE_THREAD_DEBUG_EVENT:

    break;

    case EXIT_PROCESS_DEBUG_EVENT:

    break;

    }

    return m_pSymInterface->ContinueFromCallback(fOutOfBand);

}

>> Which version of the CLR is running in the debuggee?

I'm using the CLR version returned from GetRequestedRuntime() for creating a process and GetVersionFromProcess() for DebugActiveProcess()

- Dan.

Which thread in the debugger is the one calling Continue? ICorDebug does have a rather frustrating restriction that you can'tcall Continue(OutOfBand=false) on the thread that dispatches the ICorDebugUnmanaged::DebugEvent callback. This looks like the issue you're hitting above. In .Net 2.0, I'd expect you get a CORDBG_E_CANT_CALL_ON_THIS_THREAD failure on Continue if you tried.

Random trivia re your handler:
- It's dangerous to clear all exception events because that will interefer with exceptions that the debuggee expects to have thrown. A simple adjustment would be to check the exception code and only clear Breakpoints.
- I think your LoadDll needs to close a file handle in the debug event.

FWIW, the latest MDbg sample has managed C# wrappers for the native debugging API, which you may find useful.

"I'm using the CLR version returned from GetRequestedRuntime() for creating a process and GetVersionFromProcess() for DebugActiveProcess()"
What exact version string is that? "v1.0.3705"? "v1.1.4322"? "v2.0.50727"?

Not only that you can, but you have to. Asa very quick prototype, you can have your Continue just spin up a new thread each time, and that threadProc just calls ICorDebugProcess.Continue(oob=false). Obviously, having a new thread each time is not efficient, but it will let you quickly check whether this is the problem.