Dark Bit Factory & Gravity
GENERAL => Projects => Topic started by: frea on November 26, 2007
-
Hi!
as some might remember i was trying to create a 4k demo on linux. i failed ;P, but meanwhile i created this, a quite small implementation of shadowmaping using multiple lights with GLSL. the whole code compiled & compressed is a little less that 4k ( and could be less ). Mabye it is not so impressive as creations of others( the colours are horrible ;d ), but i hope someone will find this implementation handy. I tried to explain as much of code as i could but i am not good at explaining things.
If you have problems with seeing the shadows uncomment a part of fragment shader ( colour[1]*shadow(...) ).
The code is almost platform independant, for non-linux os'es, you need to change the asm code in terrain.c to seed = time( 0 );.
( code uses GL/GLU/SDL libraries )
Could someone test it? It works ok on my comp, but always someone has a comp on which my programs fail ;).
The code is avaible here:
students.mimuw.edu.pl/~mp248778/4k.tar.gz (http://students.mimuw.edu.pl/~mp248778/4k.tar.gz)
Ps. On 64-bit os'es the file will be much bigger ( ~600-700 bytes ).
Ps2. it could be much smaller ( ~600b ) if ATI drivers had no problems when i don't load stdlib. is there a walkaround?
edit:
Windows binaries ( not striped, packed, anything ) Pressing q exits. Download nof.exe if you see only white terrain.
http://students.mimuw.edu.pl/~mp248778/1light.exe (http://students.mimuw.edu.pl/~mp248778/1light.exe)
http://students.mimuw.edu.pl/~mp248778/2lights.exe (http://students.mimuw.edu.pl/~mp248778/2lights.exe)
students.mimuw.edu.pl/~mp248778/nof.exe (http://students.mimuw.edu.pl/~mp248778/nof.exe)
students.mimuw.edu.pl/~mp248778/SDL.dll (http://students.mimuw.edu.pl/~mp248778/SDL.dll)
-
I don't have linux installed here, but looking at your code it looks a very nice job :)
-
Same as Rbraz, I have no Linux so cant run it but it looks interesting.
Why do you say ATi drivers have a problem when there is no stdlib? Is this a linux only issue. What happens?
The shaders can be smaller but they are very cool as an example, can you maybe upload an image of the result for thos without Linux but who would like to understand the code more?
Taj...
-
ATI's fglrx calls varius functions like sprintf/strlen/etc. If i compile with gcc -c (files) (flags ), and then link with ld -dynamic-linker /usr/lib64/linux-x86-64.so ( gl/sdl libraries ) ( .o files ), the program crashes when fglrx tries to call any stdlib function, which are not linked by my program.
taj : could you give ma a hint how to make these shaders smaller?
And about linux, i said that the code should be platform independant ( except that small thing in terrain.c, which is explained how to fix ).
I'll try to compile it on windows ( although i'll not try to make it 4k ;p ) and link the executable.
edit: the links to executables are in the first post.
-
taj : could you give ma a hint how to make these shaders smaller?
Try reducing all variable names to one letter. Reduce 0.5 to .5 and similar for 1.0 etc. Depending on your packer you may find it shorter to have a for loop with inline code than a function call (which has loads of overhead string-wise). I have no linux size compression experience except a really dumb self written unpacker using gzip once so I cant say for sure, but if you are using gzip to compress, it uses a dictionary so the above advice will hold true.
Hmmm the windows exes just draw a white surface (ie no shader running) under XP SP2, ATi x600...my guess is the shader at this line:
if(abs(shCo.s)<abs(shCo.q)&&\
abs(shCo.t)<abs(shCo.q)&&\
shCo.p>0)\
should be
if(abs(shCo.s)<abs(shCo.q)&&\
abs(shCo.t)<abs(shCo.q)&&\
shCo.p>0.)\
Also:
shCo=shCo*0.5f+0.5f;\
Should have no 'f's. Similar with return 1.0f;
Infact I'm confused how this works for you! Maybe "f" has crept into the standard without me noticing!
Taj
-
and karma++ for code...
-
When i reduced the variable names the size increased by 1-2 bytes.
About the f from 1.0f, the glsl specification that i have expicitly says that a float function must return a float ( no casting is allowed ), and 1.0 is a double.
If this exe wont work for you i'll just create this exe with shaders beeing loaded from file and errors written to a file, so any corrections could be much easier :).
students.mimuw.edu.pl/~mp248778/nof.exe (http://students.mimuw.edu.pl/~mp248778/nof.exe)
I hope it will work ;). It is hard to debug through a forum :P.
( surprisingly the nof version works also for me, i must have an old glsl specification ).
edit: btw under linux compile without the --short-doubles.
-
The new exe works...what did you do, make the changes I suggested?
Looks cool BTW. Very nice effect. I can see why you like it.
When i reduced the variable names the size increased by 1-2 bytes.
About the f from 1.0f, the glsl specification that i have expicitly says that a float function must return a float ( no casting is allowed ), and 1.0 is a double.
Its not a double. There are no doubles in glsl. Are you confusing with C notation? This isnt C : its glsl.
Floats are written 1.0, 0.5 .5 etc. No "f". That your ATi compiler breaks the spec and allows "f" under linux does not mean it will work everywhere. That your spec says a float function must return a float has no bearing on how a float is written in glsl.
The slight increase in size is baffling...this certainly wouldnt work like that under crinkler but I guess crinkler must be more sensitive to variable size. I'd still advise trying getting rid of your function.
-
Yes i made the changes you suggested.
you are right about these floats/doubles :). Thanks.
it is nothing new, that ATI drivers under linux work weridly :).
Is anyone now experiencing any problems with the exe's?
-
Hmmm the more I dig the more I think the code is interesting. This is a great first post, I hope you stick around. Also thanks for crediting me in the code ! I thought nobody did that anymore :-). I shall be sure to greet you when I use this stuff because I will if I can. I need to convert to VC++ and get rid of sdl first but I might even try that tonight. Good work.
(lets see if we can get it below 2k...)
Hmmm. and interesting. You are using GCC. I notice you dont use wglGetProcAddress anywhere for the extensions. Did you cross-compile under Linux for windows? Or how else did you compile. If it seems like a dumb question, its not. If I have to use
wglGetProcAddress for all those extensions you use, the exe will be much bigger. If GCC avoids this ... hmmm then maybe switching to GCC for 4ks would actually save some bytes for extension heavy code.
Taj
-
To compile it under windows i used nothing fancy, just downloaded GLee and included it into the project. That is why it is so huge btw.
gcc under linux links me to some 'advanced' ogl functions cheaply, allowing me not to use linux equivalent of wglgetprocaddress ( i forgot how is that function called ). The only problem is to find equivalents of missing functions ( like glCreateProgram(), which is glCreateProgramObjectARB() ), but it is just matter of googling / looking into libGL.so.
It might be that only my drivers give me such posibility, but i don't have any other card to test it ( anyone with nVidia card is willing to test it under linux? ). xorg-x11 drivers allow this compilation but they are not supporting shaders.
i would really like to see it fitting 2k, especialy under linux where you cannot use crinkler :).
-
...
Is anyone now experiencing any problems with the exe's?
Hmm, your windows binaries isn't working here, I got only a white rotating thing and a black background :(
gfx card: GeForce 6600GT
-
all three of them? even the nof.exe?
-
All 3 work here 8600GT/Vista.
It's a lovely demo, and I think I can see the effect, but I don't think you've chosen the best model to demonstrate it. Is it possible to put some monoliths casting shadows on there or something?
Jim
-
To add anything i would need some more space, but codewise there is no problem with that. So if taj will succed with making it smaller that will be possible. For now i am a little tired with that program, and my studies are taking more and more of my time :), so i'll try later optimizing this code.
-
To compile it under windows i used nothing fancy, just downloaded GLee and included it into the project. That is why it is so huge btw.
i would really like to see it fitting 2k, especialy under linux where you cannot use crinkler :).
Hmm well GLEE cant be used under windows as you know, so thats going to be tough to load all those extensions - there are so many. However, its such a cool piece of code I think I'll give it a go.
Taj
-
All three give me a blue screen. "2 lights" worked yestarday(28/11).
8800gts/vista
-
This is weird, especialy 2lights which worked yesterday and are not working today. I don't know what might be causing a blue screen, i am not using windows too much. I could understand a crash of the program, but a blue screen? I don't do any lowlvl stuff in that program, i just call gl functions.
-
Frea,
I converted the code from GLEE+SDL to a pure windows version. Using exactly your code plus crinkler using stupid flags) its 2492. I had no idea GZIP was so much worse than crinkler. For you and anyone interested in compiling and playing under windows using VC++ I'm putting the archive here. There is also an exe which may work better than the GLEE/SDL one or not...dont know. I have no idea why Pauls PC would Blue Screen!
Theres quite a few bytes can be saved but I dont know if I can be bothered, mainly I'm just interested in understanding this code. We'll see.
WARNING: due to size limitations I was unable to include crinkler in this distribution so it wont work "out of the bag". Crinkler is in the next post. Drop it into the directory with all the code.
Taj
-
and crinkler (called link.exe - dont worry thats how it should be called...)
-
Frea,
I think I spotted a problem in your shader code that may be causing the problems people have : cant be sure but its a hunch.
const char* vertex = "uniform mat4 lp;\
varying vec4 fragLocation;\
varying mat4 colour;\
void main()\
{\
fragLocation=gl_Vertex;\
gl_Position=ftransform();\
for(int i=0;i<2;i++)\
{\
vec3 lp=normalize(lp[i].xyz-gl_Vertex.xyz);\
float att=dot(lp,normalize(gl_Normal));\
colour[i]=vec4(1-i,i,0,0)*att;\
}\
}";
This looks dangerous as you define lp twice, once as a matrix and once as a vec3. It may be some of the shader compilers are not happy with this.
Taj
-
oh my god :D. I am surpriced now that it works at all :).
i'll check your exes a little later.
edit: btw, i am amazed by the size using crinkler ;).
The exe works for me, but i am unable to use your project file, although this is somewhat normal for my copy of vc :P. I am even unable to create an empty project, don't ask me why. I've got an legal version btw.
Seems that i will need to or write my own crinkler for linux, which would be a terribly hard task for me, or start programing under windows, which is even worse ;P.
-
oh my god :D. I am surpriced now that it works at all :).
i'll check your exes a little later.
edit: btw, i am amazed by the size using crinkler ;).
My exe still includes that bug, here is a new one without it...2 lights, also I reduced the code by about 160 bytes. I'm sure if I worked at it, it could go to 2k but I'm getting tired right now.
Yeah I'm amazed too how much better crinkler is. I'll have a lot more respect for linux 4ks in future.
Taj
-
Crinkler's not just compressing an exe and putting an unpacker on the front like gzip - it completely rewrites your executable format down at the segment level, eliminating redundant segments and overlaying matching ones, while also compressing relocs, data, and code (some clever algorithms to rewite code to get as many 0s as possible). It's mightlily impressive since not only is it packing, it's also able to behave like a normal linker.
Jim
-
all three of them? even the nof.exe?
Yes, all 3 version doesn't work here including that 2 ones by Taj, black background and withe rotating thing :(
Seems to me that there's something incompatible with Nvidia cards.
-
all three of them? even the nof.exe?
Yes, all 3 version doesn't work here including that 2 ones by Taj, black background and withe rotating thing :(
Seems to me that there's something incompatible with Nvidia cards.
Nvidia really dont do GLSL right, they focus on CG more. I think the first Nvidia cards to get glsl right are series 8XXX. That said, this is most of the skill in shaders, getting it to work on all cards out there, it can be quite tough. I have my suspicion that another bug or two lurks in this shader - its a very sophisticated one and easy to miss something.
-
even more bugs? :<.
Anyway i just discovered that p7zip is much better, on my 64 bit machine it got from 4.8 kb down to 4.1. Although still more that 4kb.
But hat means i need to test all the tricks again, mabye now something will allow me get better results. That would be nice if i would get it down to 4kb with that 600-700 bytes penalty ;).
-
even more bugs? :<.
Sorry I should be precise. "This shader and program use so many features of GLSL that its possible that even if its is correct, many cards which do not fully implement the standard will not be able to run it".
For example 2x shadow maps in one shader.
Long if condition wrapping a return
Two returns from one function
Multi texturing with shadow maps
>256 instructions (maybe, dont know)
Without Nvidia hardware here it would be hard to debug all these possibilities. Also I'm worried about the mutitexturing - if it is set up correctly , I've heard, but never done it, that shaders are VERY fussy in this area.
Taj
-
Uh. i am used to much to 100% bullet proof solutions then :).
I thought that GLSL was way more a 'standart'.
Anyway, using p7zip i was able to get down with that demo to 4095 bytes on a 64bit machine. Woohooo! ;). but i needed to change the window size to 512x512, i lacked the two last bytes :).
btw in the code the calls to glPopMatrix(), glDepthFunc() and glEnable( GL_TEXTURE_2D ) are unessesary. Also the call to switching the framebuffer to 0 in setUpShadows() is uneeded.
-
Frea,
I've worked a bit on this and got it down to 2171 bytes from the original 2492 using crinkler. For sure you can use a lot of the techniques in here (loads of opengl calls cut out). It behaves as before though slightly (slightly) slower.
Crinkler says your biggest problem now is the generate terrain function. I've had a go at at and cut it down but fundamentally its big (ąround 500 bytes). Anyway I've tried to comment what I've done but watch out becuase not every change is commented. I dont know how much of thsi will be useful under linux but for sure some of it is.
I can see loads of optimisation if you just do one light, not two of course. More than you might think actually, not just the obvious. However I didn't want to alter what your program does.
Anyway, code included, do as you will. I'm disappointed I didnt get it under 2k in windows but the terrain code is expensive although very nice. I've never seen terrain done like that before.
Taj
-
The main thing that i wanted were multiple lights :). On the begining i wanted use 4, but when i saw how 2 look i just understood that noone would see anything in there.
I'll check your code now, hope i'll learn something. I hope i spot all the changes :).
edit: when i changed sin to sinf i got few bytes on linux, how it is on windows?
To bad i didn't check the default texture parameters, i would save many bytes :)
btw why changing the random number generator into a function saved bytes? i tried it too but i got bigger size.
edit2: why does it take SO MUCH memory? 200 mb!
-
The main thing that i wanted were multiple lights :). On the begining i wanted use 4, but when i saw how 2 look i just understood that noone would see anything in there.
I'll check your code now, hope i'll learn something. I hope i spot all the changes :).
edit: when i changed sin to sinf i got few bytes on linux, how it is on windows?
To bad i didn't check the default texture parameters, i would save many bytes :)
btw why changing the random number generator into a function saved bytes? i tried it too but i got bigger size.
edit2: why does it take SO MUCH memory? 200 mb!
sinf is necessary under gcc but under vc++ sin is overloaded so , as long as floats get passed in it will use the float version.
RNG: its smaller for me, much smaller. Did you manage to spot the other optimisations in that routine (like & instead of %, no min, only max etc.)
crinkler uses 200mb and doesnt free it afterwards.
Taj
-
i spotted the min/max and &. The & was done automatically by gcc.
this behaviour of crinkler would explain why some demos that required 100mb ram crashed for me. 200 mb or crinkler + 100 mb of demo + windows + ... was more then my ram :).
edit: why did you get rid of fbo array? Is it always so that ith fbo is represented by just i ?
edit2: the only thing that i could get in terrain was to split the if( ( i < 0 ) || ( i > n1 ) ) into if( i < 0 ) i = 0; if( i == m ) break;
Nothing more ;).
-
edit: why did you get rid of fbo array? Is it always so that ith fbo is represented by just i ?
Well I got rid of it because it costs bytes :-). I am assuming fbos work like textures. If true then there is no ith FBO just as there is no "ith" texture. You can use any arbitrary int to represent a texture and I assume you can an FBO too. All the function does is guarantee to return a unique int in the texture case, I hope its the same with the FBO case.
-
In the terrain code:
If you change the terrain to use a pointer to the top left of the parabola, and a width (always 2r) and stride (n1-2r) then you might be able to save. Clip the pointer (max(heights, height_ptr)), width before drawing the parabola.
OR
To eliminate clipping altogether, render to a square which is much bigger than your landscape (you need to allocate at least r more pixels round the edge, or, if it's a texture, just go to the next power of 2 and change the uvs when you render, or use glTexSubImage.
Jim
-
1st one. if i do in that way, the calculation of the height of parabola gets more complicated( especialy if using only one pointer ). the problem is rather the amount of loops not the values they go through.
2nd i don't get it. In this code i use textures only as shadow maps. I could make the heights array bigger, but that would eliminate only two if's.
Commenting out those two if's gets me about 10b of diffrence, so not that much :).
btw taj, for me the whole terrain generation code is about 300b ( compressed ). interesting that crinkler compresses that worse than p7zip.
-
btw taj, for me the whole terrain generation code is about 300b ( compressed ). interesting that crinkler compresses that worse than p7zip.
I was quoting uncompressed - simply because I cant tell how much compression you would get. Its still the largest single routine in the code before or after compression with crinkler.
-
Only suggestions :) 10b compressed is another 10bytes...
Jim
-
Only suggestions :) 10b compressed is another 10bytes...
Jim
Absolutely true.
-
K++ to Taj and Frea.
The example programs are fantastic to look at :)
-
one way to reduce size is to remove the terrain equation completely and use one of my routines instead :P
anyways, nice work indeed..
-
um? which ones?
-
oh i have a ton of them.. ^^ how big is yours before compression?
and since i hear you speaking of parabolas i assume you generate your terrain with a method similar to this one: http://www.robot-frog.com/3d/hills/index.html
so, since they can be generated in both 2d and 3d quite easily, which method do you use? are you working on a 2d hightmap and converting to 3d, or are you simply generating triangles directly?
if you use the latter approach you are probably better off as you are, since i have absolutely no clue how to generate real 3d terrain, most of my work is in the procedural 2d department :)
-
um? which ones?
If you are prepared to use any algorithm, I reckon you could get this smaller, Minds right.
For example, you could use my old "perlin noise" hack to create a terrain...code is here:
http://in4k.untergrund.net/index.php?title=Aulds_Noise_Algorithms
I'm not 100% sure it will be smaller, lots of OGL calls you arent using already, but on the plus side, no maths - it would be under crinkler but you dont have that luxury.
I have an (as yet :-) unpublished cheat for this to make it even smaller. You dont create the random array (avoiding all random number code) but simply use a pointer to memory as input to the texture algorithm. I used this in Spheres dream and it works ok once you find a good pointer (try a pointer to the terrain function itself).
Lastly I know there is a way to write Mid point displacement which is smaller than the circle or my algorithm. Unfortunately a quick look on the web doesnt turn it up and I lost all my graphics books when I moved countries :-(. MPD is (or can be) entirely integer so it can be quite small.
This all said, I like the way yours looks _very_ much. It looks better than midpoint in my opinion.
Taj
-
Nvidia guys (and anyone else failing with older exes),
can you try to run the attached exe and tell me what errors are reported? This should help us to nail down whats up.
Thanks in advance.
Taj
-
I'm Nvidia guy :P
I got this error messages:
-
tbh, one of the easiest ways of creating a decent random terrain is just filling a 8x8 or 16x16 array with randoms and scaling it up using an algorithm somewhat more advanced than linear interpolation. anything goes here really.. experiement with a sin curve, a catmull rom splinte or cubic interpolation.. shoule be cheaper per definition.. all ofcourse dependign on how you go about implementing it :)
-
rbraz, thanks. Ah bollocks..I posted wrong binary, I'll goive it one more go tonight. Sorry for wasting your time. I'm an idiaot.
MIND! good to see you posting again - hope I get to see more of your stuff Texture-San.
-
Rbraz or someone with Nvidia, could you try this one please and report any errors...
-
I got this error, might be easy to fix I guess...
-
rbraz, thanks. Ah bollocks..I posted wrong binary, I'll goive it one more go tonight. Sorry for wasting your time. I'm an idiaot.
MIND! good to see you posting again - hope I get to see more of your stuff Texture-San.
aww <3
i havent been doing much coding lately though.. lack of inspiration i guess.. ohh but i did some ROI-coding the other week and its pretty funky stuff, i'll make sure to post some pics and explain how it's done, later :D
-
Rbraz or someone with Nvidia, could you try this one please and report any errors...
First two message requester pops up in fullscreen mode with no text in it at all. Then
it shows a white morphing landscape (sorry did not follow this thread). Test system
has a GeForce 76oo NT installed.
Hope this helps ??? Didnt expect to find any win executables in a linux thread ::)
-
Well after Bennies comments, I'm not so hopeful but this should fix your first problem Rbraz. If its white after this then I think its Nvidia having trouble with more than one shadowmap sampler : in which case this program will never work on Nvidia (less than 8XXX series).
Anyway, worth a try...
-
Whatever you did master taj - it works like a charme now.
No message requesters anymore - instead a smooth rotating landscape
with very cool lightning !!!!
Looks awesome!
-
Now it's working fine!
That's really cool effect
-
Brilliant thanks guys, karma coming your way for the help.
Frea I'm including the archive below with the shaders fixed etc. This version works on ATi and Nvidia.
WARNING: dont forget to add crinkler into directory...
Taj
-
Thanks :). Going to read these.
-
Previous one (debug.zip) ran on my Vista 8600GT after displaying two empty message boxes and a 3rd one complaining about 2 uninitialised vars in the frament shader.
Latest one NVgood is lovely :)
Jim
-
That terrain looks gorgeous!