Using LMDB / autoconf problems / Berkeley DB problems

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Using LMDB / autoconf problems / Berkeley DB problems

Nico Williams
0) Running tests under valgrind I see some warnings with Berkeley DB5
   that I haven't yet checked to see if they are problematic or not, but
   it'd be better if there weren't any.

1) I looked into testing with LMDB, but cf/db.m4 is pretty busted.

   The obvious fixes to cf/db.m4 work, but then the tests need help
   because they assume Berkeley-specific DB file naming.

   I haven't yet tested Heimdal w/ LMDB with valgrind because I haven't
   fixed the tests yet.  I hope it comes out clean and then we can start
   recommending the use of LMDB instead of Berkeley DB.

2) The fixes for cf/db.m4 are obvious but not utterly trivial.
   cf/db.m4 wants to find *one* of db3, db1, lmdb, and NDBM.

   Instead it should check and enable each backend type independently.

3) Also, NDBM should just be removed.  I see no reason to keep it, it's
   untested, probably unused (right?), and for all I know might not even
   compile.  We ship SQLite3 anyways, so it's not like NDBM is our last
   best hope if Berkeley DB and LMDB are not available...

   If once I complete (2) NDBM fails to build I'll remove it no
   questions asked.

Nico
--
Reply | Threaded
Open this post in threaded view
|

Re: Using LMDB / autoconf problems / Berkeley DB problems

Harald Barth-2

> it's untested, probably unused (right?)

That other DB backends than default are like that might be because the
documentation how to change DB backend is ... sparse.

Harald.
Reply | Threaded
Open this post in threaded view
|

Re: Using LMDB / autoconf problems / Berkeley DB problems

Nico Williams
On Mon, Jun 01, 2015 at 10:47:30AM +0200, Harald Barth wrote:
> > it's untested, probably unused (right?)
>
> That other DB backends than default are like that might be because the
> documentation how to change DB backend is ... sparse.

Lack of testing and lack of documentation are both symptoms of being
unfinished.

I'm quite interested in having a backend that is safe.  The Berkeley DB
PANIC thread has me concerned.

Nico
--
Reply | Threaded
Open this post in threaded view
|

Re: Using LMDB / autoconf problems / Berkeley DB problems

Nico Williams
In reply to this post by Nico Williams
On Mon, Jun 01, 2015 at 01:46:49AM -0500, Nico Williams wrote:
>    I haven't yet tested Heimdal w/ LMDB with valgrind because I haven't
>    fixed the tests yet.  I hope it comes out clean and then we can start
>    recommending the use of LMDB instead of Berkeley DB.

I have now, and indeed, no scary warnings.
Reply | Threaded
Open this post in threaded view
|

Re: Using LMDB / autoconf problems / Berkeley DB problems

Henry B Hotz

On Jun 1, 2015, at 9:50 AM, Nico Williams <[hidden email]> wrote:

> On Mon, Jun 01, 2015 at 01:46:49AM -0500, Nico Williams wrote:
>>   I haven't yet tested Heimdal w/ LMDB with valgrind because I haven't
>>   fixed the tests yet.  I hope it comes out clean and then we can start
>>   recommending the use of LMDB instead of Berkeley DB.
>
> I have now, and indeed, no scary warnings.

Cool!

Personal email.  [hidden email]



Reply | Threaded
Open this post in threaded view
|

Re: Using LMDB / autoconf problems / Berkeley DB problems

Howard Chu
In reply to this post by Nico Williams
Nico Williams wrote:
> 0) Running tests under valgrind I see some warnings with Berkeley DB5
>     that I haven't yet checked to see if they are problematic or not, but
>     it'd be better if there weren't any.

Most of the uninitialized memory warnings in a default build of BDB are
benign, it simply doesn't initialize every byte of every memory page it uses.
You can compile with a particular -Doption to force full initialization, but I
don't think it's usually done. (There's a performance cost.)

> 1) I looked into testing with LMDB, but cf/db.m4 is pretty busted.
>
>     The obvious fixes to cf/db.m4 work, but then the tests need help
>     because they assume Berkeley-specific DB file naming.
>
>     I haven't yet tested Heimdal w/ LMDB with valgrind because I haven't
>     fixed the tests yet.  I hope it comes out clean and then we can start
>     recommending the use of LMDB instead of Berkeley DB.

LMDB has no such problems. Most of the time LMDB doesn't even malloc any
memory, it's all being continuously reused and fully initialized each time.

--
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/
Reply | Threaded
Open this post in threaded view
|

Re: Using LMDB / autoconf problems / Berkeley DB problems

Nico Williams
On Tue, Jun 02, 2015 at 12:31:11PM +0100, Howard Chu wrote:
> Most of the uninitialized memory warnings in a default build of BDB
> are benign, it simply doesn't initialize every byte of every memory
> page it uses. You can compile with a particular -Doption to force
> full initialization, but I don't think it's usually done. (There's a
> performance cost.)

But branches dependent on unitialized memory.. that's not OK.

Nico
--