I also had the same issue when trying to find wii-tools, so I went with wiiscrubber instead which I found to be easier to find and did what I wanted it to do. I use NLZ77, but I imagine it functions similar to BatchLZ.
~When to use the compressor~
Essentially, once you extract out the .cms file you want from the Radiant Dawn ISO (such as FE10Data.cms, I'll use this as the example) you open that file up into the compressor you have, then you decompress it into a .gba file (I name it FE10Data.gba). You can check to see if you did it right by opening FE10Data.gba into the nightmare program and using the modules. If it is incorrect, you'll see the numbers are all messed up such as -127, 201, etc. If done correctly, for example, Ike's base stats should be HP:6, STR:15, MAG:1, SKL:15, SPD:12, LCK:14, DEF:12, RSE:5. After you make the changes you wanted to make, you take FE10Data.gba and open it up into you compressor and compress it back to FE10Data.cms.
~When to use WindHex~
WindHex does the same things as a nightmare module, and much, much more. For instance the Nightmare modules for Radiant Dawn can only allow you to edit base stats and growths of characters and classes, and the maximum stats for classes (as those were the only ones made). If you want to edit items, class skills, and so on, you'll need to do so in WindHex as they do not have a nightmare module made for them.
~How to Navigate WindHex~
Here's what you need to know. The left side of the WindHex shows a long list of numbers like 00000010, 00000020, and so on. At the bottom you'll see Dec: XXX Hex: XX Bin: XXXXXXXX Offset XXXXXXXX [XXX%] Asc 'A'=41. To the right is stuff I don't get, and the middle is the hex you'll be editing.
VincentASM mentions a bunch of pointers at the start of each chapter that indicates where the data starts, this is the offset. Class data starts at 0x926C, this means 0000926C in offset terms. To find this point, scroll down to the point where it says 00009260, then move to the right until you see the offset say 0000926C.
That's basically all I know with this stuff, hope that helps.