context negotiation performance problem

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

context negotiation performance problem

Eric Mei
Hi,

I searched archive but didn't find the answer.

I made a simple test for security context negotiation of MIT krb5-1.3.5,
under Linux, driven by GSSAPI: client call gss_init_sec_context() and
send token to server, server call gss_accept_sec_context() and send back
the result token.

on a P4 1.6G machine, the server only could handle ~10 requests per
second, and there's lots of disk I/O during running. By strace, I found
krb5 libraries created a temp file under "/var/tmp", and call following
syscalls for each request:
    write(fd, data, 50);
    fsync(fd);
The fsync() certainly lead to low performance. Anybody know why it is here?

Besides the fsync(), vmstat shows ~200 blocks write per seconds. It
seems the above "write(fd, data, 50)" can't produce that much data. So
where those write come from? Are there any other known issue might
affect performance?

Our server might experience 1000s of context negotiation requests at the
same time, so the above performance problem looks serious to us.

Thanks a lot for any help!

Eric
________________________________________________
Kerberos mailing list           [hidden email]
https://mailman.mit.edu/mailman/listinfo/kerberos
Reply | Threaded
Open this post in threaded view
|

Re: context negotiation performance problem

Kenneth G Raeburn
On Nov 2, 2005, at 16:03, Eric Mei wrote:

> I searched archive but didn't find the answer.
>
> I made a simple test for security context negotiation of MIT  
> krb5-1.3.5, under Linux, driven by GSSAPI: client call  
> gss_init_sec_context() and send token to server, server call  
> gss_accept_sec_context() and send back the result token.
>
> on a P4 1.6G machine, the server only could handle ~10 requests per  
> second, and there's lots of disk I/O during running. By strace, I  
> found krb5 libraries created a temp file under "/var/tmp", and call  
> following syscalls for each request:
>    write(fd, data, 50);
>    fsync(fd);
> The fsync() certainly lead to low performance. Anybody know why it  
> is here?

That's almost certainly the replay cache file, meant to prevent the  
re-use of a transmitted authenticator by an attacker.  The fsync is  
there to handle the problem of the machine crashing or losing power  
after an authenticator has been accepted and a data exchange  
performed, but the on-disk cache not yet updated.

If your protocol or threat model is such that replay attacks are not  
a problem, then under later versions of the code (I'd check whether  
it's in the 1.4 series, but the machine where I keep my source trees  
checked out ate its root file system over the weekend) you could set  
the environment variable KRB5RCACHETYPE to "none" before starting the  
program.  (Look for src/lib/krb5/rcache/rc_none.c to see if you've  
got the support.)

> Besides the fsync(), vmstat shows ~200 blocks write per seconds. It  
> seems the above "write(fd, data, 50)" can't produce that much data.  
> So where those write come from? Are there any other known issue  
> might affect performance?

Doesn't ring a bell offhand...

> Our server might experience 1000s of context negotiation requests  
> at the same time, so the above performance problem looks serious to  
> us.

Is this a single-process server, or multiple processes?  Multiple  
processes would probably deal better with the fsync calls.

Ken
________________________________________________
Kerberos mailing list           [hidden email]
https://mailman.mit.edu/mailman/listinfo/kerberos
Reply | Threaded
Open this post in threaded view
|

Re: context negotiation performance problem

Eric Mei
Ken Raeburn wrote:

> That's almost certainly the replay cache file, meant to prevent the  
> re-use of a transmitted authenticator by an attacker.  The fsync is  
> there to handle the problem of the machine crashing or losing power  
> after an authenticator has been accepted and a data exchange  
> performed, but the on-disk cache not yet updated.
>
> If your protocol or threat model is such that replay attacks are not  
> a problem, then under later versions of the code (I'd check whether  
> it's in the 1.4 series, but the machine where I keep my source trees  
> checked out ate its root file system over the weekend) you could set  
> the environment variable KRB5RCACHETYPE to "none" before starting the  
> program.  (Look for src/lib/krb5/rcache/rc_none.c to see if you've  
> got the support.)
>
> Is this a single-process server, or multiple processes?  Multiple  
> processes would probably deal better with the fsync calls.

Thanks a lot Ken. 1.4.2 indeed support KRB5RCACHETYPE=none. I enable it
and rewrite the test as multi-processes. Now the server could handle at
least 600/s now.

Thanks again!

Eric
________________________________________________
Kerberos mailing list           [hidden email]
https://mailman.mit.edu/mailman/listinfo/kerberos
Reply | Threaded
Open this post in threaded view
|

Re: context negotiation performance problem

Nicolas Williams
On Wed, Nov 02, 2005 at 04:10:34PM -0700, Eric Mei wrote:

> Ken Raeburn wrote:
>
> >That's almost certainly the replay cache file, meant to prevent the  
> >re-use of a transmitted authenticator by an attacker.  The fsync is  
> >there to handle the problem of the machine crashing or losing power  
> >after an authenticator has been accepted and a data exchange  
> >performed, but the on-disk cache not yet updated.
> >
> >If your protocol or threat model is such that replay attacks are not  
> >a problem, then under later versions of the code (I'd check whether  
> >it's in the 1.4 series, but the machine where I keep my source trees  
> >checked out ate its root file system over the weekend) you could set  
> >the environment variable KRB5RCACHETYPE to "none" before starting the  
> >program.  (Look for src/lib/krb5/rcache/rc_none.c to see if you've  
> >got the support.)
> >
> >Is this a single-process server, or multiple processes?  Multiple  
> >processes would probably deal better with the fsync calls.
>
> Thanks a lot Ken. 1.4.2 indeed support KRB5RCACHETYPE=none. I enable it
> and rewrite the test as multi-processes. Now the server could handle at
> least 600/s now.

Be sure this is safe...

What service name are you using?  If you're using 'host' then make sure
you're not also running telnet/rlogin/ftp services.  If it's something
else, and/or used only by your application then make sure that the
application protocol is such that replay protection by the mechanism is
not necessary.

One way an app protocol does that is by having an extra round-trip where
the acceptor sends a random cookie/nonce to the initiator and the
initiator parrots it back -- all with integrity protection, at least.

Nico
--
________________________________________________
Kerberos mailing list           [hidden email]
https://mailman.mit.edu/mailman/listinfo/kerberos
Reply | Threaded
Open this post in threaded view
|

Re: context negotiation performance problem

hartmans
Actually, Nico, can telnet or rlogin replay against gssapi?  I don't
think the authenticator will work.  rlogin wants a checksum of
encryption state and some other stuff.  GSS wants an 8003 checksum.

--Sam

________________________________________________
Kerberos mailing list           [hidden email]
https://mailman.mit.edu/mailman/listinfo/kerberos