Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

I am considering hiring someone to implement GPU processing in Postgres to speed up sequential scans on a database located on a ramdisk. How much do you believe I could gain using this approach? I could buy any type of graphic card. I know that this question is really hard, but I know very little about GPU, so a very rough guess would help me a lot.

EDIT: Even though I my interest is specifically for Postgres, I could ask this question in a different way: "How large is the potential performance increase by using GPU for sequential scans on an in-memory database". If the answer is that it has a potential of 10-50 times the performance, then I can start investigating more specifically if this could be realized also in Postgres.

share|improve this question
2  
Are you sure postgres is right for you? It sounds like you want a database designed for running in-memory like TimesTen or maybe CSQL –  Jack Douglas May 18 '11 at 6:28
    
Yes I am depending on many features only available in postgres. –  David May 18 '11 at 8:57
    
Can you elaborate a bit about how you plan to "implement GPU processing in Postgres"? Are you editing the postgres source, or is there some sort of emulation layer? –  Jack Douglas May 18 '11 at 9:24
    
@JackPDouglas I would get someone to edit the postgres source. –  David May 18 '11 at 11:32
    
A sensible answer would probably need some idea of your budget. How deep are your pockets? –  Jack Douglas May 18 '11 at 11:42
show 5 more comments

3 Answers

up vote 3 down vote accepted

I stumbled on some research for sqllite that may be applicable. They reported increased performance of 20-70X for some applications

It goes without saying that your mileage may vary.

share|improve this answer
add comment

You'd better ask this question on the hackers mailing list, there has been some debate about this subject in the past. The PostgreSQL developers can tell you much more about it.

http://archives.postgresql.org/pgsql-hackers/

To get good performance using a GPU for PostgreSQL, you need some form of parallel processing in PostgreSQL. This is something PostgreSQL doesn't have at the moment (for CPU nor GPU), but Greenplum already implemented this in their PostgreSQL-product for CPU's. Parallel processing is on the ToDo list, but it looks like nobody is working on it at the moment. Ask the PostgreSQL hackers about the current status.

share|improve this answer
    
Thank you for your answer. I have edited my question based upon this answer. –  David May 18 '11 at 11:35
add comment

I don't think the sequential scan will improve much. Mainly a sequential scan is iterating each record of your table, and applies an evaluation function returning a boolean value. The records matching the condition are selected for further processing. The evaluation of the function, that could be something like value>3.14 or sin(value1) > sin(value2) has to be coded by the SQL engine as an Cuda/OpenCL kernel, and then executed in the GPU. If you evaluate an user defined function, the whole function has to be converted to a kernel. If the function accesses some other data structure, it is impossible to rewrite it as a kernel. Of course, all this conversion happens behind the scene, the "user" is only supposed to write SQL. Further, the in memory data has to be moved to the GPU memory space, as a linear memory structure, such as a real (not STLed) vector. After deducting all these penalties, depending on the type of your function, you may benefit of the superior GPU speed. So you benefit only if your function is complicated, you have a lot of data, but still the function is translatable to a kernel. Your instance of postgres shall be able to write the kernel on the fly, and then call the compiler.

On the other hand, if you want to solve a specific problem with the GPU, somebody can code it directly as a client to PostgreSQL.

share|improve this answer
    
You benefit only if your query can be parallelized on the GPU - is that what you mean by "complicated"? –  Jack Douglas Aug 22 '11 at 19:28
    
The GPU is not a processor, it cannot parse your condition string and execute it against the data. For instance, you have a select * from table where value=42 Assuming there is no index on value, PostgreSQL will perform a sequential scan. your function f:{Records}->{true,false} that compares Record.value with 42 has to be written as a GPU kernel (that's a special function) and compiled before being executed. If you call "select * from employee where ismanager(id)" and ismanager() is a complicated function that requires data access, you will never be able to run it on the GPU. –  Razvan Popovici Aug 22 '11 at 20:28
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.