Comments on: Myths on bitmap indexes

By: John Wu

John Wu — Wed, 10 Oct 2007 01:45:55 +0000

For those who are really advanturous, I have been working on something called FastBit. It is available for free under LGPL license. You may download the source code at http://sdm.lbl.gov/fastbit/src/. There are quite a bit of research work behind this software. In 2001, we found it to be about 12 times faster than the best commercially available compressed bitmap indexes, see http://crd.lbl.gov/%7Ekewu/ps/LBNL-48975.html and http://crd.lbl.gov/%7Ekewu/ps/LBNL-49627.html. There are also extensive theoretical analysis of the method as well, see http://crd.lbl.gov/%7Ekewu/ps/LBNL-49626.html. You can also see a small number of applications at http://sdm.lbl.gov/fastbit/applications.html.

By: Robert H. Goretsky

Robert H. Goretsky — Mon, 24 Sep 2007 19:53:04 +0000

I have a data warehouse, which is bulk loaded on a nightly basis. When doing the INSERTs into a table with multiple BITMAP indexes, performance slows to a crawl. However, disabling and then re-creating all of the bitmap indexes would take even longer than waiting for this slow INSERT performance. Any hints/ideas here? Thanks from Robert H. Goretsky of Hoboken, NJ

By: John Wu

John Wu — Sat, 22 Sep 2007 20:29:39 +0000

There are a lot of interest ideas in the academic research publications
, does anyone have any idea where oracle is going to adopt some of them?

By: Jaromir D.B. Nemec

Jaromir D.B. Nemec — Fri, 22 Dec 2006 11:56:01 +0000

After I encountered a bitmap index of BLEVEL 12 (see http://asktom.oracle.com/pls/ask/f?p=4950:8:::::F4950_P8_DISPLAYID:1208435961077#16907176305528) I am more care full with usage of BITMAP INDEX on dynamic data.
This was in 8i. But things are changing in Oracle sometimes more quickly that the Myths can react…

Jaromir D.B. Nemec

By: Jonathan Lewis

Jonathan Lewis — Sun, 10 Dec 2006 01:34:09 +0000

I don’t think I’ve ever heard any of these "myths" expressed in this way. However, taking them in order: 1) I have had several requests for help regarding the way in which bitmap indexes grow dramatically in size following a series of small changes – especially when commits are intermittent. Bitmap indexes BECOME large when subject to OLTP-like activity. 2) Bitmap indexes are often highly appropriate for data warehouses. Bitmap indexes are usually highly inappropriate for OLTP systems unless they are indexing read-only tables – because of the concurrency effects, and the way in which bitmap indexes can grow dramatically because during updates. 3) DML on tables with bitmap indexes is often subject to concurrency problems, and can easily run into deadlock problems because of the structure of the index entries. This can result in VERY slow processing. There are very good reasons why you should not use bitmap indexes in OLTP systems, unless the tables you index are read-only (or nearly read-only). Might I suggest that you consider taking this blog entry down, doing some more tests, and then rewriting it. Regards Jonathan Lewis

By: Karl r.

Karl r. — Thu, 07 Dec 2006 12:21:03 +0000

Due to the nature of bitmap indexes (the rowid mut be calculated) do we not run into heavy CPU Usage instead of I/O Usage having a lot of DML?

Greetings
Karl

By: Patrick Sinke

Patrick Sinke — Thu, 07 Dec 2006 12:18:24 +0000

Laurent: ah, that explains it. The case is a bit more specific than I thought. So when you insert in another session a value which is “locked” in the first session, the transaction waits for the other to complete. This of course causes massive waits which causes performance degradation. Interesting!

By: Laurent Schneider

Laurent Schneider — Thu, 07 Dec 2006 12:02:23 +0000

did you try my test on 10.2.0.3?

Session 1:
SQL> create table t(name varchar2(20), gender varchar2(10));

Table created.

SQL> create bitmap index i on t(gender);

Index created.

SQL> insert into t values (‘Marc’, ‘male’);

1 row created.

Session 2:
SQL> insert into t values (‘Mary’,'female’);

1 row created.

SQL> insert into t values (‘Jack’,'male’);

As you see, the fact that it is not convenient for OLTP is more than a myth!

Regards and thanks for starting the debate

By: Patrick Sinke

Patrick Sinke — Thu, 07 Dec 2006 11:20:48 +0000

Edwin: I performed your test on Oracle 10.2.0.2.0 and in that version it’s not an issue anymore, I can’t reproduce the . So that proves there’s still myths!
I do not claim that everything in my blog is true, or that I did cover every possible case. On the contrary! What I do want to say is: do not dismiss any “new” feature because it doesn’t work in your particular situation or database version. Test what works best in every situation and in every new version of the database. See for yourself, test and try, and choose the best alternative.

Many examples I see here are about Oracle 8.1.7 or even 8.0.3. It is not said that those problems are fixed in newer versions, but they probably are.

I think Stewart words it very well: it are not so much myths as it are half-truths. Many of those half-truths exist and it’s up to us to find out what is true and what not. Every time again. And when I read all the comments (thank you all again!) I see that there’s still a lot more to find out for me on this subject.

By: Gary

Gary — Thu, 07 Dec 2006 01:14:21 +0000

We had a look at using bitmap indexes in an OLTP application, and it was a very poor fit and we abandoned it.
Firstly we hit “Myth 1″ with the size of the bitmap indexes expanding MASSIVELY when lots of single row inserts/updates/deletes (or small number of rows) were done for the indexed columns. These single/few row changes are the bread-and-butter of OLTP databases.
Secondly, “Myth 2″. Its not so much the nature of the data, but the nature of the data processing. Data warehouse queries expect to deal with hundreds/thousands and even millions of rows (eg the orders submitted on a given day). OLTP queries expect to deal with single/several/dozens of rows (eg the items on a specific order). Bitmap indexes are used on RELATIVELY low cardinality columns, ie ones that may identify thousands of records out of millions, not ones that identify dozens of records out of millions.
Now many ‘OLTP’ databases will have some queries/reports that do deal with thousands of rows, and bitmap indexes may be appropriate there. However bitmaps aren’t really appropriate for the ‘bread-and-butter’ OLTP queries.
Thirdly, “Myth 3″. Most of the ‘poor performance’ comes from locking/concurrency issues and from the IO associated with Myth 1.

Function based indexes are a totally different issue and they are definately under represented.