John Tsiombikas nuclear@mutantstargoat.com
25 March 2014
I revisited recently a dormant project of mine, for which I unfortunately need to write a 3dsmax exporter plugin.
Now, I'm always pissed off from the start when I have to write code on windows and visual studio, but having to deal with 3dsmax on top of that, really just adds insult to injury. It's not just that maxsdk is a convoluted mess. Or that it needs a very specific version of visual studio to write plugins for it (which is really Microsoft's fault, to be fair). No, my biggest issue so far is that 3dsmax takes about 3 years to start up, and there is no way to unload, or reload a plugin without restarting it.
Whenever I fix a tiny thing in the exporter plugin I'm writting, and I want to try it out and see if it does the buissiness, I have to shut down 3dsmax, start it up again (which takes forever), load my test scene, then try to export again and see what happens. This is obviously unacceptable, so I really had to do something about it.
But first let's examine how exporter plugins are handled by 3dsmax. The 3dsmax installation directory contains a subdirectory called plugins. At startup, it looks into that directory, and loads all the plugins located there. Plugins are obviously just DLLs, but with a particular naming convention for their suffix, so that 3dsmax will know how to treat them as different plugin classes. For instance, exporter plugins must be named with a .dle suffix.
Exporter plugins are supposed to define a subclass of SceneExport, which
overrides the DoExport function. When 3dsmax loads the plugin dll, it pulls
from it a function called LibCreate, which is supposed to create and return an
instance of this class. Then, whenever the user tries to export a scene of the
type registered by this plugin, the appropriate DoExport function is called, to
do it.
So as I mentioned previously, the problem is that plugin DLLs can't be unloaded and reloaded while 3dsmax is running, making it necessary to shut down 3dsmax and restart it all the time, while writing my exporter.
After cursing Autodesk and their stupid plugin system for a while, I came up with a pretty neat idea to circuimvent that restriction. Instead of letting 3dsmax grab hold of my exporter DLL when it starts, what if I write a second exporter DLL that is just a stub, a proxy if you will. What if that stub gets loaded by 3dsmax at startup, but my actual exporter plugin DLL is separate, 3dsmax doesn't even know about it, and the stub exporter loads it up on demand, and passes through the DoExport call!
With this scheme, it doesn't actually matter much that plugins can’t be unloaded by 3dsmax, because my plugin wasn't loaded by 3dsmax in the first place. I'm loading it on demand when the user requests an export, and releasing it immediately after the export is done.
Turns out, this works beautifully. I created post-build events for both
projects (maxgoat, and maxgoat_stub) that copies the DLLs in the 3dsmax plugin
directory, but only the maxgoat_stub post-build event changes the suffix to
".dle". The actual exporter is left with the original ".dll" suffix, and 3dsmax
ignores its existence.
The only tricky bit the stub DLL has to do in order to load and use the real exporter DLL, is to find the plugin directory where both of them are installed. This little snippet of win32 barf handles that correctly:
static const char *find_dll_dir()
{
static char path[MAX_PATH];
HMODULE dll;
if(!GetModuleHandleExA(GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS |
GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNT,
(LPCSTR)find_dll_dir, &dll)) {
return 0;
}
GetModuleFileNameA(dll, path, sizeof path);
char *last_slash = strrchr(path, '\\');
if(last_slash) {
*last_slash = 0;
}
return path;
}
At that point it can happily proceed to load the actual exporter DLL, and call
everything it needs to, in order to perform the export. Here's the whole
DoExport function of the stub, with all error handling removed for brevity:
int GoatExporterStub::DoExport(const MCHAR *name, ExpInterface *eiface,
Interface *iface, BOOL non_interactive, DWORD opt)
{
static const char *dll_fname = "maxgoat.dll";
const char *plugdir = find_dll_dir();
char *dll_path = new char[strlen(dll_fname) + strlen(plugdir) + 2];
sprintf(dll_path, "%s\\%s", plugdir, dll_fname);
// load plugin and grab all the functions we need
HMODULE dll = LoadLibraryA(dll_path);
ClassDescFunc get_class_desc = (ClassDescFunc)GetProcAddress(dll, "LibClassDesc");
InitFunc init = (InitFunc)GetProcAddress(dll, "LibInitialize");
ShutdownFunc shutdown = (ShutdownFunc)GetProcAddress(dll, "LibShutdown");
init();
ClassDesc *desc = get_class_desc(0);
// create the exporter instance and call DoExport
SceneExport *ex = (SceneExport*)desc->Create();
int res = ex->DoExport(name, eiface, iface);
delete ex;
shutdown();
delete [] dll_path;
FreeLibrary(dll);
return res;
}
With this system in place, iterating on figuring out all the maxsdk bullshit is a breeze. Just keep 3dsmax open with my stub plugin loaded all the time, compile/install the exporter at will, and hit export again from within 3dsmax. The current version of the exporter DLL is automatically loaded, used and shoved out of the way again. The whole process becomes almost enjoyable.
This was initially posted in my old wordpress blog. Visit the original version to see any comments.