Great Masters of the Necropian Abyss: I SUMMON THEE!!!

[eyesodilated]

Apr 1, 2023 10:40:28 GMT -6 ozz said:

The fix for mirroring is just changing one variable.
The first byte in sequence data is something used for animation looping I think (like how it repeats a few frames during long spell casts), while the second byte is for mirroring.

Yes that is correct, doing it in HEX as you said would pose a challenge as that variable needs to be changed for every frame of every sequence for 3 out of the 5 directional facings. Theoretically with a new LOMUT we can fix this for sure! That is now the ultimate goal.

But what if we have a work around in the meantime just for testing where we "fix" the .imp in hex editing manually. I have been trying to explore that while fixing lomut.c

I have some questions, if you look at the RLE everything looks fine until I get to the u4 PaletteOffset, the bytes seem reverse. They should be a28 but instead the bytes read 28 0A at that offset. I notice alot of the bytes are reverse like this for the offsets like they are written in big endian? IDK its wierd. 



Also have you tried editing the hex code to test? When you manipulate the hex code how are you rehashing the checksum?

[ozz]

You are confusing the two. In little endian the least significant byte comes first, which is the case for all variables here. Always read them from last to first.

It is 0A28, just like the previous variables are 0053 and 0052 (not 5300 and 5200).

[eyesodilated]

Here is the binary template for liwiza.imp, it is built specifically for liwiza.imp and is only usable in 010 Hex Editor:

liwizaIMP.bt (80.54 KB)

It is a byte by byte rundown of the entire RLE for liwiza.imp based on imp.c variable field enumeration. But it is a caveman tool nonetheless. I would like to make a .bt that works for all .imp files however 010 Hex editor uses a custom script that looks like Python and C++ had a baby. I even made a C Program to parse any .imp file and store the offsets in a CSV file along with the values. It seems, because of my very basic skills, that I will have to make a seperate parser for each of the five functions of imp.c in the .imp RLE, that being the Header, Sequence, Facing, Frame and Hotspot. I also isolated the Palette Data and put it in the binary template. The Sequence field array gets created based off the u2 NumSequence value from the Header. The Facing array size also uses this value to determine it's own size, basically 5 multiplied against the NumSequence value- this will tell you how many structs to loop. The Frame and Hotspot array gets it size from the Size value of the Facing struct. The Frame struct is at top near header, the Hotspot struct appears to seek the end of the file, go back 16 bytes, before going back the number of bytes in the Hotspot array variable enumeration, again given by imp.c so I might continue to work on these IMP structure parsers, seems easier and faster than learning the binary template custom script 010 Hex uses. Here is the code if anyone wants to mess with it, I keep it and the imp file in C:\LOMUT FYI:

impheader.c (3.3 KB)


I can compile into an executable maybe in a different post since I will be stitching together a number of parsers into a single program or run them from a different interface or even a GUI. 

So I think this is the way we are going to finally fix all the issues. I want to modify lomut.c but right now editing the RLE byte data seems the most promising and best solution. Not only that but I have been looking at .smp and .smk files to get them to work in game or modify them out of game via hex. I have some issues though. I have still been unable to get a fully working sprite in the game this way. Changing bytes defintely affects the hotspot. But I may be doing it wrong, I have tried copy and pasting, setting values to zero for hotspot field. We need to figure out what is going on here. Maybe the last 16 bytes of every file are a Checksum for the Hotspot struct which is worrying me. I have some evidence of this look:


Although I admit I could be losing my mind staring at this code the chances that these 16 bits of data add up to a number in the billions and that number directly translates to a hex code that is equal to the Checksum for 32 bit variables seems like a mathmatical miracle. But this is Lord's of Magic so I am not suprised and have looked at several .imp and can't reproduce the checksum match. I might need to disassemble LoMSE to see if there is any Checksum calls. At least I have a guess of the checksum variable location and we know the algorithm so we can reverse engineer a way to "rehash" the value. Why I think there is a checksum is because I have been unable to modify the Hotspot values and load the .imp into the game without the game crashing. Changing the Width and Height in the Header, (which are the max values for width and height out of all the frames), yields glitched results. Changing the Frame data also gives me glitched results. Changing both also glitches out. Changing both and the Hotspot data definitely glitches out here is what I mean:







As you can see the sprite has been spaghettified, the runs are all over the place. Note this is actually a simplified test sprite and custom .H file that has only 5 frames of Stand animation so I can test it quickly. This water dungeon has a nice grid I've been using to test. Even on a brand new .imp compiled from LOMUT where the initial images are 60 by 60 LOMUT still puts in a byte value of 512 for the Width and Height in the header. Hopefully we dont need to going into the 1 byte RLE pcx.c structs to solve this, but I am conifdent we dont need to do that, the variables in imp.c should be enough. I think in this test sprite since it is an custom sprite even if I change the 512 * 512 values in the header the rest of the RLE seems to be encoding on based off that value, so changing it changes where the runs of pixels start in the RLE. So I may need to make another program that can calculate the Hotspot DispX, DispY and HotX and HotY as well as the Frame struct's width and height. These old RLE run from top corner left to right. So I have had success getting in a rectangle that is perfect size and the hotspot works!!! But it is a rectangle not a sprite, maybe I should start moving my new sprites to the top corner of the 512 * 512 image? That may work but I am not sure what other problems arise.

The only thing I can 100% modify is the palette data with no issue. If I modify the PaletteOffset(the first 1) I think this changes with what Faith the unit is on. In an attempt to create Fire Lizards I changed the Palette offset but it seems like Fire Lizards on the mauruders party had a different color red! Also jsut straight up editing the palette data, which is in there right after frame with a 1 byte padding if odd number Sequences/Frames. Anyway the palette data is just there in HTML format ARGB I think is the order encoded. Anyway this lead to real neat palette swaps that have no HotSpot errors!!! Is this how you made Frozen Shades Boaster???

Anyway great work guys we are SO CLOSE! What do yall think any ideas/solutions? Maybe I overlooked something? 010 Hex has a free 30 trial to use the .bt if you want to try yourself:



Here is the formula for using the structs to determine Hotspot area:
Type: This indicates whether the frame is a normal frame or a duplicate-back-reference frame. A duplicate-back-reference frame means that it is not stored as a separate frame, but instead it is a reference to a previous frame. This can save space in the file.

HSType: This specifies the type of value that the HSpot variable holds. It can hold either a packed 16-bit XY displacement or an offset to a HotSpot structure.

Width and Height: These variables represent the real width and height of the frame in pixels.

Size: This variable represents the compressed size of the frame, or the uncompressed size if the frame is a duplicate or a type 9 frame. It should be aligned to 4 bytes.

HSpot: This variable holds the Hot Spot data for the frame. It can either hold a packed XY displacement or an offset to a HotSpot structure.

Offset: This variable holds the offset to the frame data or the number of the duplicating frame.

About the HotSpot structure. It contains information about the Hot Spot of a frame, which is the pixel location that is used as the anchor point when the frame is displayed. The HotSpot structure contains the following variables:



DispX and DispY: These variables represent the XY displacement from the virtual Width and Height. They indicate how far the Hot Spot is from the top-left corner of the virtual image.

HotX and HotY: These variables represent the click-point for cursors. They indicate where the user should click if they want to interact with the frame. For units, HotX and HotY may represent the spawning location of the missile or the location of the unit's feet.

Unknown variables: These variables are currently unknown and their purpose is not clear.

To visualize this data in pixel terms, you can use the following formula:



PixelX = DispX + HotX

PixelY = DispY + HotY

This will give you the pixel coordinates of the Hot Spot for the frame.

So maybe there is no Checksum maybe I need to figure out to handle HSType and HSpot as well as the Offset... We can see the promised land folks!

[ozz]

If HSType is anything other than 0, then HSpot will contain an offset to the relevant hotspot data (that is, the location of the data in the file).
The hotspot data at that offset (usually near the end of the file) will contain sets of 3 1-byte variables, times the number in HSType, with padding of zeroes in the end (to align to 8 bytes).
In each trio you will have:
Type: Refer to my previous post for what it each value means.
X offset.
Y offset.

The struct for hotspots in imp.c is incorrect. It assumes that the hotspot data is always of the same length (2 sets), when it can be of any length.

As you can see, there are also hotspots of types 3, 7 and 9.

...and yes, it does seem that cursor hotpot is used as sort of an anchor for the rest of them.

[eyesodilated]

Apr 21, 2023 13:51:11 GMT -6 ozz said:

If HSType is anything other than 0, then HSpot will contain an offset to the relevant hotspot data (that is, the location of the data in the file).
The hotspot data at that offset (usually near the end of the file) will contain sets of 3 1-byte variables, times the number in HSType, with padding of zeroes in the end (to align to 8 bytes).
In each trio you will have:
Type: Refer to my previous post for what it each value means.
X offset.
Y offset.

The struct for hotspots in imp.c is incorrect. It assumes that the hotspot data is always of the same length (2 sets), when it can be of any length.

As you can see, there are also hotspots of types 3, 7 and 9.

...and yes, it does seem that cursor hotpot is used as sort of an anchor for the rest of them.

Thankyou I can now see that simply editing the hotpsot data was not quite enough, if you look at the original liwiza.imp compared to a decompiled then recompiled one all the HSType values are no longer 2 but 0! I tried many combinations but I did not see that this value was affected. So the game was reading my recompiled and hex edited .imp with a HSType of 0 so the game was looking for a packed 16 bit DispX and DispY instead of the Offset to the Hotspot structure!


  This gives me hope , pretty cool how the Offset address is in the u4 Offset of the Frame struct. Another great find!



My next question now is where did you get the 3 1-byte variables? I am finding that all 8 variables in the HotSpot struct are signed 2 byte variables: Unknown1, DispX, DispY, Unknwon2,HotX,HotY, Unknown3, Unknown4.


But now I have new ideas to tinker with, I need to fix the HSType variable so the game knows what to do! Thanks! Sorry ive been absent lately, in the middle of moving again- Should have some tests done soon so we can confirm

[ozz]

Yes, you are correct, they are 2 bytes. My mistake.

In any case - unknown1 and unknown2 are the types of the hotspots, while unknown3 and unknown4 are padding. That is assuming that the hotspot data is 2 sets and not more (that is, HSType isn't more than 2). That struct only works for that particular size.

Does it even put any hotspot data in at all after you recompile it? It's not like it can get it from anywhere at that point?

[mushka]

At MLSDev, our dedicated development team provides expert consulting and software design services. We specialize in creating dynamic web and mobile applications, custom software solutions, and integrated systems that meet the needs of our clients. Our experienced developers are proficient in modern technologies like artificial intelligence (AI), machine learning, natural language processing, computer vision, and predictive analytics. We understand that businesses have unique challenges and strive to provide customized solutions that will maximize their potential. Our goal-oriented team works closely with each client to help them reach their objectives mlsdev.com/services/dedicated-development-team.

[Boaster]

Is anyone considering making some new building sprites or doodads?

[eyesodilated]

May 17, 2023 8:40:30 GMT -6 ozz said:

Yes, you are correct, they are 2 bytes. My mistake.

In any case - unknown1 and unknown2 are the types of the hotspots, while unknown3 and unknown4 are padding. That is assuming that the hotspot data is 2 sets and not more (that is, HSType isn't more than 2). That struct only works for that particular size.

Does it even put any hotspot data in at all after you recompile it? It's not like it can get it from anywhere at that point?

It puts in the either the value 119 and 127 for all hotspot values but if I'm off one byte these values are actually 00 and 00. Sorry been gone a while had a second job thought I needed to move but may stay here on this lease now.
Pretty sure it's 119 and 127 for all recompiled imps however it being 0 and 0 makes more sense! I should be able to bring my laptop with me to nightshift now that I'm down to one job.

We are very close to a working solution! We are in the endgame 

[eyesodilated]

Jun 10, 2023 6:49:25 GMT -6 Boaster said:

Is anyone considering making some new building sprites or doodads?

I have definitely tinkered with this idea, got a working building sprite and building interior into the game. I feel like when I added custom dungeon and encounters building sprites or doodads into the game I got some wierd results.
What did you have in mind?

[Boaster]

Jun 24, 2023 8:26:08 GMT -6 eyesodilated said:

Jun 10, 2023 6:49:25 GMT -6 Boaster said:

Is anyone considering making some new building sprites or doodads?

I have definitely tinkered with this idea, got a working building sprite and building interior into the game. I feel like when I added custom dungeon and encounters building sprites or doodads into the game I got some wierd results.
What did you have in mind?

I didn't have anything in mind. I was merely asking if anyone has thought to create any? I know there's a lot going on here about unit sprites and such. So, I wondered about the others.

[eyesodilated]

Yes I have a list of stuff I want to ask you about eventually when I get too stuck- building sprites and dungeon graphics at top of that list. I was trying to add a new building for my water quest. I sacrificed a pre-existing dungeon building sprite to make a labyrinth building for my Minotaur "boss" encounter. I got it in and even attempted to animate it (like the flags and some dungeon sprites like bloody fountain death has)-but had some palette issues mostly. That plus I sacrificed the original dungeon. I have always dreamed of making towns and villages More strategically important to me.i wanted to have a special building be available after certain conditions. THat way I can find newly created units like mermaid or werewolf etc and train/hire them. That way I don't need to use cheat code to get them in. Maybe even building allows forging of artifacts. I get a village now and maybe I create a Lighthouse that looks awesome but I rarely use rather conquer the capital if I want off faith units. What other doodads are there? I messed around with the cursor overworked but it didn't look good. Also been looking at projectiles and spell sprites too. I've been saying this almost 10 years but we are close to being able to do what we want with sprites, our only obstacle will be time as always. Side note I've been having an awful time getting new .wav and .smk files into the game and actually work as intended. Maybe these still make a call.to the disk? Maybe I haven't searched the gs.mpq enough and am missing a call or command somewhere. 

[Ceriax]

It's so cool that after all this time, people are still finding cool things to do with the game. I was reading this thread and it looks like the way the game codes for sprites is very finicky and it makes it difficult to add or change in-game graphic assets.

Have you had any more progress? I'd love to see a LOMSE mod with more available units, perhaps even with some new voices!

[Ceriax]

I'm far from an expert in coding, so I can only offer an uneducated guess.

The game seems quite receptive to numerical value changes, which is probably why adding spells and artifacts to it was comparatively much easier. It also seems to tolerate non-interactive images such as added unit portraits and artifact pictures.

Perhaps the engine of the game which controls interactive assets such as units and building sprites is mostly spaghetti code? Modern games with much more detailed assets are often modded relatively easily so it can't just be a complexity issue. I imagine the dev team who coded the game back in the 90s weren't thinking "Hey, a small group of people 25 years from now might want to add a bunch of custom unit types so let's make the base code of the game highly accessible."

[eyesodilated]

Mar 2, 2024 19:53:19 GMT -6 Ceriax said:

It's so cool that after all this time, people are still finding cool things to do with the game. I was reading this thread and it looks like the way the game codes for sprites is very finicky and it makes it difficult to add or change in-game graphic assets.

Have you had any more progress? I'd love to see a LOMSE mod with more available units, perhaps even with some new voices!

I have some new sprites i have made. Ive rewatched Game of Thrones and I am working on a complete mod almost all brand new units.it would have a new legends of urak quest. But thats a per project. Im willing to make new sprites that fit into the original artwork and theme. 
Sound files and the video .smk files i use to be able to edit. Its not been working lately.
Another user has made some Python scripts that could be the key to fixing all the issues once and for all.