Editing GLSL shaders with PSPad editor

I quite like PSPad editor, which is free, quite complete and quite robust.

I use it a lot for editing things and others, including GLSL shaders.

I drafted an (incomplete for now) syntax highlight configuration file for GLSL shaders for PSPad, you can grab it here.

It also allows to enable the code explorer for shader files, thanks to the "C-like" structure, nice!

Nothing fancy, just trying to catchup with news

FWIW, trying to catchup with news - after my vacations and the all-announces-while-SIGGRAPH rush :

• OpenGL 3.2 specification is available, along with Open(G|C)L BOF slides.
  
• NVidia announced their OptiX realtime raytracing API / library, which will be available in early autumn
• AMD made publicly available their OpenCL 1.0 (Khronos conformance stamp seems pending) CPU implementation in their ATI Stream SDK 2.0 beta.
• Papers, talks and slides are being linked from here, some commented here - so far, I quite liked the synthetic Parallel Graphics in Frostbite presentation, still catching up.

Assembly 2009 also took place, while I didn't like the winner demo, I enjoyed the invitation demo and the '3rd place' demo (links here and here) - seminar videos are also available.

I also stumbled upon ramenhdr project which seems to use Adobe/Boost GIL to make HDR image compositing - if I feel like having another headache, I should take a look.

building with Scons & MSVC8 using PCH with PDB and /Zi

Some note concerning my current attempts to evaluate Scons 1.2 for my needs to replace my 'pure' Visual Studio solutions builds.

After trying to automatically convert my solutions to Scons scripts, which did not work at all, I decided to start from scratch.

I easily got the bare build and link done on one of my modules. And I tried to gradually introduce 'features' in my build : namely Precompiled header (PCH) support, and Debug information generation.

Adding both was easy, using the Scons man page as a reference, I used a construct like this one :

env['PCHSTOP'] = "precompiled.h"

env['PCH'] = env.PCH( os.path.join(builddir, 'precompiled.cpp') )[0]

env['PDB'] = os.path.join( builddir, "%s.pdb" % BASE_NAME )


Using this setup, it works fine out-of-the-box, but Scons msvc tool uses the /Z7 compiler flag to create the debug information. This has the effect to store debug informations in every built obj file, and according to Scons man page (at CCPDBFLAGS information), allow parallel builds to work fast & correctly.

This duplication of debug information lead my, simple, build directory to grow from about 33Mb (using my existing reference MSVC vcproj to build) to about 60Mb, almost 100% bigger...

I took a look at using /Zi instead of /Z7, to have a single PDB containing my debug information, and it took some to figure out how to have it work with PCH altogether.

According to Scons man page, all you have to do is ::

env['CCPDBFLAGS'] = ['${(PDB and "/Zi /Fd%s" % File(PDB)) or ""}']

Using this setup, I had two problems :

• the linker complained about LNK4203 errors, preventing debugging information to be linked.
• when doing a partial build of the project, the compiler complained about a C2859 error, indicating that the PDB file was not the one generated with the PCH, and instructing me to rebuild all.

In fact, what happens, is that Scons instructs the compiler to issue debug information in a given PDB file (say A.PDB) and also instructs the linker to output the debug information in A.PDB. My supposition is that the linker first starts to trash the PDB, and hence cannot find the debug information (LNK4203 errors). Afterwards, the A.PDB file is the one generated by the linker, and the compiler cannot use it anymore with the PCH...

I changed the PDB flags to instruct the compiler to use an intermediate PDB file :

env['CCPDBFLAGS']  = ['${(PDB and "/Fd%s_incremental.pdb /Zi" % File(PDB)) or ""}']

And now everything works fine.

A few performance tests on sequential file read methods under Windows

After reading Timothy Farrar's post on I/O thread scheduling, I thought that I made my latest performance tests on File I/O (especially read) too long ago.

I took some time to build a few no-brainer test functions, reading a file of about 1Gb, all read requests are 64Kb.

I essentially tested different a few variations :

• STDIO - standard stdio streams
• STD_S - standard stdio streams + 'sequential' microsoft extension (fopen(filename, "rbS"))
• WIN32 - Win32 API CreateFile using FILE_ATTRIBUTE_NORMAL
• WINNB - Win32 API CreateFile using FILE_FLAG_NO_BUFFERING
• MAPPB - Mapped file input (unrealistically mapping the whole file, and lazily using boost::iostreams::mapped_file)

As I'm not currently interested in latency, but more in brute-force throughput, I did not bother tested any async I/O.

On the same file, on the same hard drive (Seagate ST3250410AS), on the same machine, I ran the test on an XP32 and a Vista64.

To make it short, timings follow, it seems that :

• under XP32, using FILE_FLAG_NO_BUFFERING is worth the hassle. For lazy people, using Microsoft extension to fopen ("rbS" mode) looks like an easy good win.
• under Vista64, memory mapped file is the slowest approach, and all others compare almost equal, with a win for fopen in "rbS" mode.

I made quick tests on other machines, and the trends are the same.

XP32

Y:\>bench_disk_access_code.exe f:\ortho1.megatex
STDIO f:\ortho1.megatex : 33.35 Mb/sec. - 995754.00 Kb
WIN32 f:\ortho1.megatex : 33.57 Mb/sec. - 995754.00 Kb
STD_S f:\ortho1.megatex : 68.37 Mb/sec. - 995754.00 Kb
WINNB f:\ortho1.megatex : 87.09 Mb/sec. - 995754.00 Kb
MAPPD f:\ortho1.megatex : 79.23 Mb/sec. - 995754.00 Kb

Vista64

Y:\>bench_disk_access_code.exe f:\ortho1.megatex
STDIO f:\ortho1.megatex : 86.45 Mb/sec. - 995754.00 Kb
WIN32 f:\ortho1.megatex : 86.94 Mb/sec. - 995754.00 Kb
STD_S f:\ortho1.megatex : 88.42 Mb/sec. - 995754.00 Kb
WINNB f:\ortho1.megatex : 86.82 Mb/sec. - 995754.00 Kb
MAPPD f:\ortho1.megatex : 76.58 Mb/sec. - 995754.00 Kb

Parsing OpenGL 'spec' files

A few notes after some experiments in parsing OpenGL spec files, describing enums and commands for every OpenGL version and extension.

The parsing part when smoothly, as the files are consistently formatted, but I encountered some traps in interpreting values and commands though.

'gl.spec'

It describes OpenGL commands. Every command has a 'category' property that mostly represent the extension or GL version it belongs to. For exemple, OpenGL 1.2 commands have a 'VERSION_1_2' category value, and so on.

Problems come from GL 1.0 & 1.1 commands, that require special handling :

• GL 1.0 commands have a 'category' which truly represent a category within GL functionnalities (such as 'drawing', 'display-list', ...) and not an extension or GL version.

• GL 1.1 commands have a 'category' value of '1_1' (should be 'VERSION_1_1' to be consistent) which is easily fixed.

'enumext.spec'

It contains every enum value - starting from OpenGL 1.2... A comment in the file mentions that previous values are not needed as vendors provide a gl.h file containing every entries for OpenGL 1.1... doh.

Otherwise, interpretation is quite smooth. Enum values are grouped in 'sections' that represent either a GL_VERSION or an extension name.

Care must be taken to handle value aliases, as the aliased values may be defined after the alias definition (such as COMPARE_REF_TO_TEXTURE aliasing to COMPARE_R_TO_TEXTURE_ARB) or not defined in the file, when aliasing to OpenGL 1.1 (such as CLIP_DISTANCE0 aliasing to CLIP_PLANE0)... doh.

'enum.spec'

It contains every enum definition, for OpenGL 1.0 to the latest extension, and it used to allocate enum values for vendors.

It looks like it's not really meant to be parsed in its whole : every extension definition is in a commented section, without obvious formal grammar. The parsing of the uncommented lines allow to get the definitions for OpenGL <= 1.1 enums, that are missing from 'enumext'.

The parsing has a lot in common with 'enumext' except that, here, the 'sections' are the GL commands accepting the enum value as a parameter. So the set of every defined value (not in comment) should represent the whole OpenGL <= 1.1 enums. It seems that there is, at least, a typo in the spec file that prevents this good behaviour : TEXTURE_BINDING_3D is defined in 'enum.spec' but it belongs to OpenGL 1.2, as seen in 'enumext.spec'.

Merging enums

To resolve the enums values, without missing values or duplicates (i hope), I set up a linker-like behaviour :

• scan 'enum.spec' to create the GL1.1 value table (including some wrong values)
• scan 'enumext.spec', using the previous value table to resolve external aliases
• remove from 'enum.spec' value table every enum defined in 'enumext.spec'