@relsoft: the ISO contains what is on the PSP game CD. Usually in there you have a couple of folders that are necessary to boot up the game and then a huge file, which is the game archive. Not all the games have the same structure though.
The game files can be compressed, although this does not always happen because there is enough room on the CD for the game. Most of the time, the game data is just aggregated with the code in the archive.
What I did for a start was to open the huge binary with a hex editor. I usually start with this method to detect visually the structure of the binary. In this case, one can quickly observe that the huge file is a id/chunk structured file. there are 4-bytes signatures for chuncks followed by what could be sizes for the chunks or offsets. Very similar to file formats derived from EA-IFF on the Amiga.
While looking further in the file, I observed portions that looked like a binary mess, usually where the game code is. Then large (even huge portions) with more regular patterns, usually where the graphics are... and suddenly I stopped in a region that looked like a knight's helmet! That was the first hint for a game like Ghosts'n Goblins, wasn't it?? I saved that part of the file as a RAW image and opened it in Photoshop. Indeed, it was a knight's helmet. I then continued in the binary file and found something that looked like a sword or a stick. This time it wasn't just luck. I was simply looking at some of the bitmaps in the file. I did the same thing with Photoshop, and yes, it was a sword. Colors were not right, but the shape was there.
Then I looked again a few bytes before the beginning of the portion where the helmet's data was and found a signature that said "PIC" => gotcha! Looked like I found the header of a bitmap file. I compared that with the data portion where the sword was, and found the same thing! Then I started to look at the bytes after the signature and found some classic numbers there: 32, 64, 128 etc... classic texture dimensions for a console game.
After this small success I wrote a small ripper that looked for the signature and extracted the RAW data using these magic numbers. I ran the small piece of code and my little exe started to spit dozens of files. Some worked, some not. I got helmets, potions, weapons, wall textures, etc.
The files that did not work, were of two kinds. Some were just a mess of pixels, others were almost ok, but not perfect. However these almost perfect bitmaps reminded me a lot of RGB16/RGB15 encoding used in the Targa file format. Again I tried to look at the bytes near the file header and yes, I found that there was an extra byte that was set on some specific value for the images that were ripped fine, another value for those that worked almost and a third value for those that did not work well.
Again I refined my code and tested the possible case of RGB16/RGB15 encoding. And... it worked... I was now getting over a thousand textures.
I then started to look on the net about PSP file formats and did not find much, except some hacked tools to change the PSP themes. There I got more information about the picture file formats. I improved my ripper and also observed that in the package, there were other file formats, like PNG, true TGA and the PSP format GIM (GIM... IMG.. IMaGe.. well intuition worked well).
I then searched over the internet and gamer forums about all this... very little information... but found a link on a chinese website. All was in chinese, but the re was a brief description of the GIM header. Byte offsets. But the description was in chinese. It was time to use some online translation to reveal some secrets. And secrets were revealed. The guy that posted that on the internet was apparently working for Sony in China. He must have been part of the team in the test dept of Sony's factory... That became really exciting. The translation was very poor of course, but the website mentioned a zip archive that was part of the SDK for the PSP, where the format was described! Of course no link of that SDK.
I then searched for that zip archive and found two interesting links. One on the Sony's PSP website that was talking about the SDK for game developers and briefly described the contents of it. One could request to become an official PSP developer to obtain the SDK, but besides the fact that you had to pay for it, Sony does not guarantee you that you will become an authorized developer. Their decison is really up to them and they could simply refuse your request because they do not see a fit between you and their goals! Great, forget about Sony!
The second link was leading me to a russian website! Ahaaaa... That was easier to translate! The website mentioned that someone, somewhere had the zip archive I was looking for... because he was part of a team that worked on a game for the PSP! Classic thing... it happened during the Amiga day... game developers were former hackers and often the information got out of secrecy through these obscure pipelines.
I then started to seach for the zip archive, the handle/nickname of the "famous" source and a few porn sites later found a hidden page with the zip archive! Bingo! I opened the zip file and heard the heaven's bells ringing! It was the actual C headers that described the file format!
Back to my little ripper, I started to improve it and found that the bitmap files were using the texture format required by the graphics pipeline on the PSP. Several references in the header files gave me links on the Homebrew PSP websites. There I found about the PSP internals and data formats. That helped me a lot and I perfected my ripper. I learned that there was a 3D object file format, the GMO. But that is another story...