Kerberos transport DNS record design

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Kerberos transport DNS record design

Greg Hudson
Currently we can map from a Kerberos realm name to the TCP and/or UDP
addresses of a KDC, kpasswd, or kadmind server.  Several of us have been
discussing how to best extend that mechanism to include MS-KKDCP, and at
the same time minimize the number of DNS lookups required to contact a
realm without configuration.

We had been planning to use the URI record type, but after a recent
round of discussion, we don't think it's likely that popular DNS hosting
providers will allow customers to create URI records (since it seems
like no one else is using them).  Some middle-boxes would also block DNS
queries for URI records.  That problem would be even worse if we create
a new record type.  So, we are planning to use the TXT record type.  It
seems unlikely that we can standardize on a TXT record through the IETF
(except perhaps as an informational RFC), but it seems like the only
deployable option for individuals and small organizations.

If we use the TXT record type, we have to define the payload format,
which is the primary subject of this thread.  The payload must
communicate:

* Priority, because DNS records are unordered
* Transport type--currently one of TCP, UDP, and MS-KKDCP
* For the TCP and UDP types, the hostname and optional port
* For the MS-KKDCP type, the proxy URL

The format must be extensible to future transport types, including
ones that use TCP, UDP, or HTTP in different ways.  Examples could
include Heimdal's HTTP proxy protocol or RFC 6251 (Kerberos over TLS).

That leaves a lot of questions:

* Do we want to include weights as well as priorities?  I think weights
  are unnecessary complexity, even just as an optional field in the
  format, but I seem to be in the minority.

* Do we want to include master KDCs in the same query as normal KDCs, or
  should they be in a separate record?  (Master KDCs are used as a
  fallback by the MIT client code when we receive a KDC error which
  could have resulted from out-of-date data due to propagation delays.)

  - If we do communicate master KDCs in the same record, should it be
    possible to exclude a master KDC from the normal server list (so
    that it is only contacted if a normal KDC returns an error), or is
    it sufficient to be able to assign master KDCs a low priority?

* Do we want to make our payload isomorphic to the URI payload, in
  anticipation of migrating to the URI record type in the future?  I
  would argue that such a migration is vanishingly unlikely.  If we go
  this way, the payload contains a priority, a weight, and a URI, so we
  have to encode everything but the priority inside a URI, opening a
  bunch of other inter-related questions:

  - Should we register a new URI scheme to represent TCP and UDP
    transports, or use the unregistered tcp: and udp: schemes as some
    other applications have done?

  - Should we use a new URI scheme (probably the same one as above) for
    MS-KKDCP, or should we just use the https: URI of the proxy?

  - If we do create new URI schemes, should we use use separate schemes
    for krb5/kpasswd/kadmin, or use the same scheme since we already
    know the application protocol going in?

  - Should we use fragment identifiers (#suffix) to indicate master-ness
    and/or the transport type, or should we use a prefix?

  If we don't restrict ourselves to an isomorphic-to-URI payload format,
  we still have to decide whether and how to represent master KDC
  entries, but the other concerns largely go away.

* At the bikeshed level, what should we use as a separator between
  fields?  Is it more convenient for DNS administrators if we avoid
  using whitespace as a separator?
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Nathaniel McCallum-5
On Wed, 2016-05-25 at 17:48 -0400, Greg Hudson wrote:

> Currently we can map from a Kerberos realm name to the TCP and/or UDP
> addresses of a KDC, kpasswd, or kadmind server.  Several of us have
> been
> discussing how to best extend that mechanism to include MS-KKDCP, and
> at
> the same time minimize the number of DNS lookups required to contact
> a
> realm without configuration.
>
> We had been planning to use the URI record type, but after a recent
> round of discussion, we don't think it's likely that popular DNS
> hosting
> providers will allow customers to create URI records (since it seems
> like no one else is using them).  Some middle-boxes would also block
> DNS
> queries for URI records.  That problem would be even worse if we
> create
> a new record type.  So, we are planning to use the TXT record
> type.  It
> seems unlikely that we can standardize on a TXT record through the
> IETF
> (except perhaps as an informational RFC), but it seems like the only
> deployable option for individuals and small organizations.
>
> If we use the TXT record type, we have to define the payload format,
> which is the primary subject of this thread.  The payload must
> communicate:
>
> * Priority, because DNS records are unordered
> * Transport type--currently one of TCP, UDP, and MS-KKDCP
> * For the TCP and UDP types, the hostname and optional port
> * For the MS-KKDCP type, the proxy URL
>
> The format must be extensible to future transport types, including
> ones that use TCP, UDP, or HTTP in different ways.  Examples could
> include Heimdal's HTTP proxy protocol or RFC 6251 (Kerberos over
> TLS).
>
> That leaves a lot of questions:
>
> * Do we want to include weights as well as priorities?  I think
> weights
>   are unnecessary complexity, even just as an optional field in the
>   format, but I seem to be in the minority.

It is my strong preference to include weight so that implementations
which already have weight processing logic can utilize this
information. MIT does not need to support it unless clear justification
presents itself.

> * Do we want to include master KDCs in the same query as normal KDCs,
> or
>   should they be in a separate record?  (Master KDCs are used as a
>   fallback by the MIT client code when we receive a KDC error which
>   could have resulted from out-of-date data due to propagation
> delays.)
>
>   - If we do communicate master KDCs in the same record, should it be
>     possible to exclude a master KDC from the normal server list (so
>     that it is only contacted if a normal KDC returns an error), or
> is
>     it sufficient to be able to assign master KDCs a low priority?

If the only reason to specify a master KDC is as a last-ditch fallback,
then priority should be sufficient.

> * Do we want to make our payload isomorphic to the URI payload, in
>   anticipation of migrating to the URI record type in the future?  I
>   would argue that such a migration is vanishingly unlikely.  If we
> go
>   this way, the payload contains a priority, a weight, and a URI, so
> we
>   have to encode everything but the priority inside a URI, opening a
>   bunch of other inter-related questions:

It is my preference to support a future migration to URI, even if we
grant that such a trasition is vanishingly unlikely. Doing so adds no
additional burden to our implementation since we already have to
process a URI for KKDCP. Put short, we will have to write a parser for
whatever format we use. Thus, I think our default should be to use a
URI format unless there is a compelling reason not to.

>   - Should we register a new URI scheme to represent TCP and UDP
>     transports, or use the unregistered tcp: and udp: schemes as some
>     other applications have done?
>
>   - Should we use a new URI scheme (probably the same one as above)
> for
>     MS-KKDCP, or should we just use the https: URI of the proxy?

I do not have any strong preference for the format. Something like this
might work:

priority:weight:krbkdc:udp:host[:port]
priority:weight:krbkdc:tcp:host[:port]
priority:weight:krbkdc:tls:host[:port]
priority:weight:krbkdc:kkdcp:http://host[:port][/path]
priority:weight:krbkdc:kkdcp:https://host[:port][/path]

>   - If we do create new URI schemes, should we use use separate
> schemes
>     for krb5/kpasswd/kadmin, or use the same scheme since we already
>     know the application protocol going in?

Is there any value in having a single query return both kpasswd and
kadmin? If so, then I think separate schemes are desirable.

>   - Should we use fragment identifiers (#suffix) to indicate master-
> ness
>     and/or the transport type, or should we use a prefix?

It is my preference to avoid fragments. Colon separation logic can't be
reused for fragments, so we just add an additional parsing burden. I'd
like to avoid this complexity.

> * At the bikeshed level, what should we use as a separator between
>   fields?  Is it more convenient for DNS administrators if we avoid
>   using whitespace as a separator?

As I said above, we already have to parse a URI for KKDCP, let's resuse
this parsing logic. I'm against whitespace separators for this reason.
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Greg Hudson
On 05/26/2016 10:39 AM, Nathaniel McCallum wrote:

>> * Do we want to include master KDCs in the same query as normal KDCs,
>> or
>>   should they be in a separate record?  (Master KDCs are used as a
>>   fallback by the MIT client code when we receive a KDC error which
>>   could have resulted from out-of-date data due to propagation
>> delays.)
>>
>>   - If we do communicate master KDCs in the same record, should it be
>>     possible to exclude a master KDC from the normal server list (so
>>     that it is only contacted if a normal KDC returns an error), or
>> is
>>     it sufficient to be able to assign master KDCs a low priority?
>
> If the only reason to specify a master KDC is as a last-ditch fallback,
> then priority should be sufficient.

Priority only affects the order in which KDCs are contacted when
higher-priority KDCs fail to answer at all.  Master KDCs are used as a
fallback if a non-master KDC returns an error like "preauth failed"
which could result from stale KDC data.

So, priority might be a sufficient alternative to "master only," but it
cannot handle the problem entirely.  If we do not include an indication
of master KDCs in the TXT record payload, we will have to make a second
query of another record to find them.  (In theory this query is only
necessary on failure, but currently the code does it whenever it gets a
KDC response.)
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Matt Rogers
In reply to this post by Nathaniel McCallum-5
On 05/26, Nathaniel McCallum wrote:

> On Wed, 2016-05-25 at 17:48 -0400, Greg Hudson wrote:
> > * Do we want to make our payload isomorphic to the URI payload, in
> >   anticipation of migrating to the URI record type in the future?  I
> >   would argue that such a migration is vanishingly unlikely.  If we
> > go
> >   this way, the payload contains a priority, a weight, and a URI, so
> > we
> >   have to encode everything but the priority inside a URI, opening a
> >   bunch of other inter-related questions:
>
> It is my preference to support a future migration to URI, even if we
> grant that such a trasition is vanishingly unlikely. Doing so adds no
> additional burden to our implementation since we already have to
> process a URI for KKDCP. Put short, we will have to write a parser for
> whatever format we use. Thus, I think our default should be to use a
> URI format unless there is a compelling reason not to.
>

+1

> >   - Should we register a new URI scheme to represent TCP and UDP
> >     transports, or use the unregistered tcp: and udp: schemes as some
> >     other applications have done?
> >
> >   - Should we use a new URI scheme (probably the same one as above)
> > for
> >     MS-KKDCP, or should we just use the https: URI of the proxy?
>
> I do not have any strong preference for the format. Something like this
> might work:
>
> priority:weight:krbkdc:udp:host[:port]
> priority:weight:krbkdc:tcp:host[:port]
> priority:weight:krbkdc:tls:host[:port]
> priority:weight:krbkdc:kkdcp:http://host[:port][/path]
> priority:weight:krbkdc:kkdcp:https://host[:port][/path]
>

No particular preference here, so this format is fine to me.

>
> >   - Should we use fragment identifiers (#suffix) to indicate master-
> > ness
> >     and/or the transport type, or should we use a prefix?
>
> It is my preference to avoid fragments. Colon separation logic can't be
> reused for fragments, so we just add an additional parsing burden. I'd
> like to avoid this complexity.
>
> > * At the bikeshed level, what should we use as a separator between
> >   fields?  Is it more convenient for DNS administrators if we avoid
> >   using whitespace as a separator?
>
> As I said above, we already have to parse a URI for KKDCP, let's resuse
> this parsing logic. I'm against whitespace separators for this reason.

+1 on both points.


> _______________________________________________
> krbdev mailing list             [hidden email]
> https://mailman.mit.edu/mailman/listinfo/krbdev

--
Matt Rogers
Red Hat, Inc
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Nathaniel McCallum-5
In reply to this post by Greg Hudson
On Thu, 2016-05-26 at 11:12 -0400, Greg Hudson wrote:

> On 05/26/2016 10:39 AM, Nathaniel McCallum wrote:
> > > * Do we want to include master KDCs in the same query as normal
> > > KDCs,
> > > or
> > >   should they be in a separate record?  (Master KDCs are used as
> > > a
> > >   fallback by the MIT client code when we receive a KDC error
> > > which
> > >   could have resulted from out-of-date data due to propagation
> > > delays.)
> > >
> > >   - If we do communicate master KDCs in the same record, should
> > > it be
> > >     possible to exclude a master KDC from the normal server list
> > > (so
> > >     that it is only contacted if a normal KDC returns an error),
> > > or
> > > is
> > >     it sufficient to be able to assign master KDCs a low
> > > priority?
> >
> > If the only reason to specify a master KDC is as a last-ditch
> > fallback,
> > then priority should be sufficient.
>
> Priority only affects the order in which KDCs are contacted when
> higher-priority KDCs fail to answer at all.  Master KDCs are used as
> a
> fallback if a non-master KDC returns an error like "preauth failed"
> which could result from stale KDC data.
>
> So, priority might be a sufficient alternative to "master only," but
> it
> cannot handle the problem entirely.  If we do not include an
> indication
> of master KDCs in the TXT record payload, we will have to make a
> second
> query of another record to find them.  (In theory this query is only
> necessary on failure, but currently the code does it whenever it gets
> a
> KDC response.)

How likely is this failure from non-master KDCs due to synchronization
issues? Is this a real historical problem? Does this problem persist
today?

Does anyone else implement this logic?
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Greg Hudson
On 05/26/2016 12:24 PM, Nathaniel McCallum wrote:
> How likely is this failure from non-master KDCs due to synchronization
> issues? Is this a real historical problem? Does this problem persist
> today?

Any site with a non-trivial replication delay can be affected by this
problem whenever users change their passwords.  It is a real historical
problem, and it persists as a concern for the MIT KDC deployment.  I
don't have a lot of visibility into other deployments, but we have
received a request within the last few years to expand the master
fallback to TGS requests.

> Does anyone else implement this logic?

I don't see any evidence that Heimdal implements it, so we may be the
only one.  It's a legitimate question how Heimdal gets away with not
implementing this fallback.  If you list your master KDC first in the
KDC list, then the frequency of kinit failures after a password change
goes way down, but I've measured as high as a 1% fallback rate in the
MIT Kerberos deployment when the master KDC is operational.

I don't think we're currently in a position to simplify out this part of
the initial creds design.  Complexity, once added, is hard to safely remove.

I forgot to comment on some parts of your initial reply:

> It is my preference to support a future migration to URI, even if we
> grant that such a trasition is vanishingly unlikely.

The main cost here is registering one or more URI schemes, unless we
decide to shoehorn our responses into existing schemes.  See RFC 7595
for the procedures involved.  It also makes the DNS responses slightly
larger since the URI scheme (completely predictable if we register our
own) needs to appear in each record.

> Is there any value in having a single query return both kpasswd and
> kadmin? If so, then I think separate schemes are desirable.

I don't think there is any value in including kpasswd and kadmin entries
in the same record.  While we do fall back from kpasswd to port-changed
kadmin when locating kpasswd servers, this is only a provision for
realms with misconfigured DNS.
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

RE: Kerberos transport DNS record design

Brandon Allbery
As someone with admin experience with both MIT and (older versions of) Heimdal:

- MIT's propagation mechanisms are somewhat limited; either you do bulk propagation at set times, which leads to changes between those times not being seen in a timely fashion on the slave KDCs, or you can use incremental propagation which only works reliably with a relatively low change rate and is prone to stalling and eventually failure as the change rate increases.
 
- It took Heimdal quite a while to get its incremental propagation (iprop) to be reliable, but in reasonably recent versions iprop does not have the stability issues under moderate to high change rates that MIT's kiprop has. As such, iprop makes master KDC fallback unnecessary in most cases.

- For either one, using LDAP backend with its replication facilities makes fallback unnecessary.

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Greg Hudson
Sent: Thursday, May 26, 2016 1:10 PM
To: Nathaniel McCallum <[hidden email]>; [hidden email]
Subject: Re: Kerberos transport DNS record design

On 05/26/2016 12:24 PM, Nathaniel McCallum wrote:
> How likely is this failure from non-master KDCs due to synchronization
> issues? Is this a real historical problem? Does this problem persist
> today?

Any site with a non-trivial replication delay can be affected by this problem whenever users change their passwords.  It is a real historical problem, and it persists as a concern for the MIT KDC deployment.  I don't have a lot of visibility into other deployments, but we have received a request within the last few years to expand the master fallback to TGS requests.

> Does anyone else implement this logic?

I don't see any evidence that Heimdal implements it, so we may be the only one.  It's a legitimate question how Heimdal gets away with not implementing this fallback.  If you list your master KDC first in the KDC list, then the frequency of kinit failures after a password change goes way down, but I've measured as high as a 1% fallback rate in the MIT Kerberos deployment when the master KDC is operational.

I don't think we're currently in a position to simplify out this part of the initial creds design.  Complexity, once added, is hard to safely remove.

I forgot to comment on some parts of your initial reply:

> It is my preference to support a future migration to URI, even if we
> grant that such a trasition is vanishingly unlikely.

The main cost here is registering one or more URI schemes, unless we decide to shoehorn our responses into existing schemes.  See RFC 7595 for the procedures involved.  It also makes the DNS responses slightly larger since the URI scheme (completely predictable if we register our
own) needs to appear in each record.

> Is there any value in having a single query return both kpasswd and
> kadmin? If so, then I think separate schemes are desirable.

I don't think there is any value in including kpasswd and kadmin entries in the same record.  While we do fall back from kpasswd to port-changed kadmin when locating kpasswd servers, this is only a provision for realms with misconfigured DNS.
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev

_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Nathaniel McCallum-5
In reply to this post by Greg Hudson
On Thu, 2016-05-26 at 13:10 -0400, Greg Hudson wrote:

> On 05/26/2016 12:24 PM, Nathaniel McCallum wrote:
> > How likely is this failure from non-master KDCs due to
> > synchronization
> > issues? Is this a real historical problem? Does this problem
> > persist
> > today?
>
> Any site with a non-trivial replication delay can be affected by this
> problem whenever users change their passwords.  It is a real
> historical
> problem, and it persists as a concern for the MIT KDC deployment.  I
> don't have a lot of visibility into other deployments, but we have
> received a request within the last few years to expand the master
> fallback to TGS requests.
>
> > Does anyone else implement this logic?
>
> I don't see any evidence that Heimdal implements it, so we may be the
> only one.  It's a legitimate question how Heimdal gets away with not
> implementing this fallback.  If you list your master KDC first in the
> KDC list, then the frequency of kinit failures after a password
> change
> goes way down, but I've measured as high as a 1% fallback rate in the
> MIT Kerberos deployment when the master KDC is operational.
>
> I don't think we're currently in a position to simplify out this part
> of
> the initial creds design.  Complexity, once added, is hard to safely
> remove.

Agreed. Would MIT consider it sufficient to do a secondary lookup in
this case? A secondary record could have the same format as the primary
record. The cost for this is, of course, a secondary lookup. However,
this only happens in the failure case.

> I forgot to comment on some parts of your initial reply:
>
> > It is my preference to support a future migration to URI, even if
> > we
> > grant that such a trasition is vanishingly unlikely.
>
> The main cost here is registering one or more URI schemes, unless we
> decide to shoehorn our responses into existing schemes.  See RFC 7595
> for the procedures involved.

Well, I'm not sure we need to register it soley for the purposes of an
informational draft. I suspect we'd only need to register if we tried
to push a URI draft through. But you and I already consider this
"vanishingly unlikely."

> It also makes the DNS responses slightly
> larger since the URI scheme (completely predictable if we register
> our
> own) needs to appear in each record.

We are only talking about seven bytes ("krbkdc:") since all the other
contents of the record are required in some form. I'm fine with
removing this if people don't see a value in it. We can always add the
scheme should a serious effort behind the URI record appear.

> > Is there any value in having a single query return both kpasswd and
> > kadmin? If so, then I think separate schemes are desirable.
>
> I don't think there is any value in including kpasswd and kadmin
> entries
> in the same record.  While we do fall back from kpasswd to port-
> changed
> kadmin when locating kpasswd servers, this is only a provision for
> realms with misconfigured DNS.

So then my preference is to avoid a separate scheme.

======================================================================

MIT implements the followin SRV record lookups:
    _kerberos-master.REALM
    _kerberos-adm.REALM
    _kerberos.REALM
    _kpasswd.REALM
    _krb524.REALM

Heimdal supports the following service lookup plugin types:
    locate_service_master_kdc
    locate_service_kadmin
    locate_service_kdc
    locate_service_kpasswd
    locate_service_krb524

However, Heimdal only appears to implement a _kerberos.REALM SRV lookup
internally. Heimdal contains a reference to _kadmin.REALM in an in-tree
copy of a previous attempt at a DNS discovery draft, however, this was
change to "_kerberos-adm" in a subsequent version of the draft:
https://tools.ietf.org/html/draft-ietf-krb-wg-krb-dns-locate-03

It thus seems to me like there is large agreement between MIT and
Heimdal. Thus, I propose the following:

1. Implement all the record names that MIT already supports as TXT.
Using exactly the same semantics that MIT already implements (i.e.
ignoring the weight parameter).

2. Make the format of the TXT record:

    priority:weight:udp:host[:port]
    priority:weight:tcp:host[:port]
    priority:weight:tls:host[:port]
    priority:weight:kkdcp:http://host[:port][/path]
    priority:weight:kkdcp:https://host[:port][/path]

This is isomorphic with the URI record with the exception of a missing
scheme, which could be added later should the desire to use URI emerge.
It also avoids the problem of defining a URI scheme now. I think we can
get away with this by noting that this format isn't technically a URI
even though it closely resembles one.

Thoughts?


_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Roland C. Dowdeswell
In reply to this post by Brandon Allbery
On Thu, May 26, 2016 at 05:25:19PM +0000, Brandon Allbery wrote:
>

> - For either one, using LDAP backend with its replication facilities
> makes fallback unnecessary.

Does the LDAP backend promise that all of the slaves are up to date
before kadm5_create_principal() returns successfully?  If not,
there will still be a race condition.  We are currently starting
to see our developers deploy new services using VMs and containers
in an on-demand fashion on a large scale expecting to be able to
hit those services the moment the master KDC reports that the
principal has been created.

I also quite like the master failover because it provides additional
resiliency against things like: a slave's clock going out of skew;
an administrator truncating the Kerberos DB on a slave; a slave
running out of disk space and somehow rubbishing the KDB causing
the KDC to respond inappropriately; or an administrator of DNS
(often people in a different group in the organisation) messing up
the one of the SRV RRs to point at a different realm's KDC.  I
suppose that I don't think that it makes sense for the client to
simply give up at the first sign of errors when there is a decent
chance that one of the other KDCs may very well have a better
answer.  Remember that in many environments, almost all requests
to the KDCs should succeed, after all: if I'm making a TGS_REQ for
a service then in general I've connected to the service in question
and it has asked me to use GSSAPI/Kerberos authentication (at least
in the case of protocols like HTTP/Negotiate, SSH, SASL where GSSAPI
auth is negotiated.)

--
    Roland Dowdeswell                      http://Imrryr.ORG/~elric/
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

RE: Kerberos transport DNS record design

Brandon Allbery
None of the backends provides a guarantee that all slaves are updated; in fact it is very likely (but not a guarantee) that *no* slave has been updated at that point, regardless of backend. LDAP and Heimdal's iprop would be fastest to update afterward, depending on configuration, and most probably LDAP can be tuned to minimize the time: LDAP does not rely on relatively long delays (measured in minutes instead of seconds) to validate lock-free update semantics, as the KDC-based replications do.

Otherwise, you do make good points about continued utility of the fallback in edge cases.

-----Original Message-----
From: Roland C. Dowdeswell [mailto:[hidden email]]
Sent: Thursday, May 26, 2016 10:59 PM
To: Brandon Allbery <[hidden email]>
Cc: Greg Hudson <[hidden email]>; Nathaniel McCallum <[hidden email]>; [hidden email]
Subject: Re: Kerberos transport DNS record design

On Thu, May 26, 2016 at 05:25:19PM +0000, Brandon Allbery wrote:
>

> - For either one, using LDAP backend with its replication facilities
> makes fallback unnecessary.

Does the LDAP backend promise that all of the slaves are up to date before kadm5_create_principal() returns successfully?  If not, there will still be a race condition.  We are currently starting to see our developers deploy new services using VMs and containers in an on-demand fashion on a large scale expecting to be able to hit those services the moment the master KDC reports that the principal has been created.

I also quite like the master failover because it provides additional resiliency against things like: a slave's clock going out of skew; an administrator truncating the Kerberos DB on a slave; a slave running out of disk space and somehow rubbishing the KDB causing the KDC to respond inappropriately; or an administrator of DNS (often people in a different group in the organisation) messing up the one of the SRV RRs to point at a different realm's KDC.  I suppose that I don't think that it makes sense for the client to simply give up at the first sign of errors when there is a decent chance that one of the other KDCs may very well have a better answer.  Remember that in many environments, almost all requests to the KDCs should succeed, after all: if I'm making a TGS_REQ for a service then in general I've connected to the service in question and it has asked me to use GSSAPI/Kerberos authentication (at least in the case of protocols like HTTP/Negotiate, SSH, SASL where GSSAPI a!
 uth is negotiated.)

--
    Roland Dowdeswell                      http://Imrryr.ORG/~elric/

_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Simo Sorce-3
The fallback makes sense in a single-master/multiple-slaves scenario, in
a multi-master scenario it doesn't really do much, as you do not know
*which* master has the new data, but you may be lucky that replication
happened by the time you try again with another server.

Simo.

On Fri, 2016-05-27 at 18:08 +0000, Brandon Allbery wrote:

> None of the backends provides a guarantee that all slaves are updated;
> in fact it is very likely (but not a guarantee) that *no* slave has
> been updated at that point, regardless of backend. LDAP and Heimdal's
> iprop would be fastest to update afterward, depending on
> configuration, and most probably LDAP can be tuned to minimize the
> time: LDAP does not rely on relatively long delays (measured in
> minutes instead of seconds) to validate lock-free update semantics, as
> the KDC-based replications do.
>
> Otherwise, you do make good points about continued utility of the
> fallback in edge cases.
>
> -----Original Message-----
> From: Roland C. Dowdeswell [mailto:[hidden email]]
> Sent: Thursday, May 26, 2016 10:59 PM
> To: Brandon Allbery <[hidden email]>
> Cc: Greg Hudson <[hidden email]>; Nathaniel McCallum <[hidden email]>; [hidden email]
> Subject: Re: Kerberos transport DNS record design
>
> On Thu, May 26, 2016 at 05:25:19PM +0000, Brandon Allbery wrote:
> >
>
> > - For either one, using LDAP backend with its replication facilities
> > makes fallback unnecessary.
>
> Does the LDAP backend promise that all of the slaves are up to date before kadm5_create_principal() returns successfully?  If not, there will still be a race condition.  We are currently starting to see our developers deploy new services using VMs and containers in an on-demand fashion on a large scale expecting to be able to hit those services the moment the master KDC reports that the principal has been created.
>
> I also quite like the master failover because it provides additional resiliency against things like: a slave's clock going out of skew; an administrator truncating the Kerberos DB on a slave; a slave running out of disk space and somehow rubbishing the KDB causing the KDC to respond inappropriately; or an administrator of DNS (often people in a different group in the organisation) messing up the one of the SRV RRs to point at a different realm's KDC.  I suppose that I don't think that it makes sense for the client to simply give up at the first sign of errors when there is a decent chance that one of the other KDCs may very well have a better answer.  Remember that in many environments, almost all requests to the KDCs should succeed, after all: if I'm making a TGS_REQ for a service then in general I've connected to the service in question and it has asked me to use GSSAPI/Kerberos authentication (at least in the case of protocols like HTTP/Negotiate, SSH, SASL where GSSAPI!
  a!
>  uth is negotiated.)
>
> --
>     Roland Dowdeswell                      http://Imrryr.ORG/~elric/
>
> _______________________________________________
> krbdev mailing list             [hidden email]
> https://mailman.mit.edu/mailman/listinfo/krbdev


--
Simo Sorce * Red Hat, Inc * New York

_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Simo Sorce-3
In reply to this post by Nathaniel McCallum-5
On Thu, 2016-05-26 at 16:45 -0400, Nathaniel McCallum wrote:

> Thus, I propose the following:
>
> 1. Implement all the record names that MIT already supports as TXT.
> Using exactly the same semantics that MIT already implements (i.e.
> ignoring the weight parameter).
>
> 2. Make the format of the TXT record:
>
>     priority:weight:udp:host[:port]
>     priority:weight:tcp:host[:port]
>     priority:weight:tls:host[:port]
>     priority:weight:kkdcp:http://host[:port][/path]
>     priority:weight:kkdcp:https://host[:port][/path]
>
> This is isomorphic with the URI record with the exception of a missing
> scheme, which could be added later should the desire to use URI
> emerge.
> It also avoids the problem of defining a URI scheme now. I think we
> can
> get away with this by noting that this format isn't technically a URI
> even though it closely resembles one.
>
> Thoughts?


I like this proposal.
Simo.

--
Simo Sorce * Red Hat, Inc * New York

_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Greg Hudson
In reply to this post by Nathaniel McCallum-5
On 05/26/2016 04:45 PM, Nathaniel McCallum wrote:
> Would MIT consider it sufficient to do a secondary lookup in
> this case? A secondary record could have the same format as the primary
> record. The cost for this is, of course, a secondary lookup. However,
> this only happens in the failure case.

We talked about this a bit on IRC.  For archival:

* Doing a second lookup for master KDCs is somewhat easier to implement
because that's how the code inside locate_kdc.c works for other
information sources.  However:

* Currently, we do the master lookup on success as well as failure
(http://krbdev.mit.edu/rt/Ticket/Display.html?id=7721).  Fixing this is
moderately difficult, requiring changes to the krb5_sendto_kdc() and
k5_get_init_creds() contracts.  Unless this is fixed, and if we use a
separate lookup for master KDCs, the TXT record feature adds two new DNS
queries for most KDC lookups instead of just one, in realms where TXT
records aren't yet deployed.

* Many realms don't have master KDCs.  Representing the absence of
master KDCs with an NXDOMAIN response poses some minor issues: it would
trigger a fallback to SRV lookups unless we take steps to avoid that)
and the TTL for NXDOMAIN caching is global to a DNS zone.  Nico notes
that we could allow an empty TXT record to represent the absence of
master KDCs to avoid these issues.

Including master KDCs in the normal KDC RR set could be done by adding
an alphabetical flags field to the TXT record format, where the 'M' flag
indicates a master KDC entry.  If we do this, we can store an is_master
value in each struct server_entry, and k5_kdc_is_master() can use that
value to avoid doing the master KDC lookup.  We would wind up repeating
the query for the TXT record if we actually do fall back (assuming no
fix for #7721), but we can hope that record is cached by a local resolver.

> 1. Implement all the record names that MIT already supports as TXT.
> Using exactly the same semantics that MIT already implements (i.e.
> ignoring the weight parameter).

I don't think we need to implement or document krb524 lookups; that's a
vestige.  We should be able remove those references from locate_kdc.c
and add a comment in locate_plugin.h that locate_service_krb524 will
never be queried in recent versions.

> 2. Make the format of the TXT record:
>
>     priority:weight:udp:host[:port]
>     priority:weight:tcp:host[:port]
>     priority:weight:tls:host[:port]
>     priority:weight:kkdcp:http://host[:port][/path]
>     priority:weight:kkdcp:https://host[:port][/path]
>
> This is isomorphic with the URI record with the exception of a missing
> scheme, which could be added later should the desire to use URI emerge.
>
> It also avoids the problem of defining a URI scheme now. I think we can
> get away with this by noting that this format isn't technically a URI
> even though it closely resembles one.

+1 except that I currently favor adding a flags field to indicate master
KDC records.


_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Nathaniel McCallum-5
On Sat, 2016-05-28 at 02:23 -0400, Greg Hudson wrote:

> On 05/26/2016 04:45 PM, Nathaniel McCallum wrote:
> > Would MIT consider it sufficient to do a secondary lookup in
> > this case? A secondary record could have the same format as the
> > primary
> > record. The cost for this is, of course, a secondary lookup.
> > However,
> > this only happens in the failure case.
>
> We talked about this a bit on IRC.  For archival:
>
> * Doing a second lookup for master KDCs is somewhat easier to
> implement
> because that's how the code inside locate_kdc.c works for other
> information sources.  However:
>
> * Currently, we do the master lookup on success as well as failure
> (http://krbdev.mit.edu/rt/Ticket/Display.html?id=7721).  Fixing this
> is
> moderately difficult, requiring changes to the krb5_sendto_kdc() and
> k5_get_init_creds() contracts.  Unless this is fixed, and if we use a
> separate lookup for master KDCs, the TXT record feature adds two new
> DNS
> queries for most KDC lookups instead of just one, in realms where TXT
> records aren't yet deployed.
>
> * Many realms don't have master KDCs.  Representing the absence of
> master KDCs with an NXDOMAIN response poses some minor issues: it
> would
> trigger a fallback to SRV lookups unless we take steps to avoid that)
> and the TTL for NXDOMAIN caching is global to a DNS zone.  Nico notes
> that we could allow an empty TXT record to represent the absence of
> master KDCs to avoid these issues.
>
> Including master KDCs in the normal KDC RR set could be done by
> adding
> an alphabetical flags field to the TXT record format, where the 'M'
> flag
> indicates a master KDC entry.  If we do this, we can store an
> is_master
> value in each struct server_entry, and k5_kdc_is_master() can use
> that
> value to avoid doing the master KDC lookup.  We would wind up
> repeating
> the query for the TXT record if we actually do fall back (assuming no
> fix for #7721), but we can hope that record is cached by a local
> resolver.
>
> > 1. Implement all the record names that MIT already supports as TXT.
> > Using exactly the same semantics that MIT already implements (i.e.
> > ignoring the weight parameter).
>
> I don't think we need to implement or document krb524 lookups; that's
> a
> vestige.  We should be able remove those references from locate_kdc.c
> and add a comment in locate_plugin.h that locate_service_krb524 will
> never be queried in recent versions.
>
> > 2. Make the format of the TXT record:
> >
> >     priority:weight:udp:host[:port]
> >     priority:weight:tcp:host[:port]
> >     priority:weight:tls:host[:port]
> >     priority:weight:kkdcp:http://host[:port][/path]
> >     priority:weight:kkdcp:https://host[:port][/path]
> >
> > This is isomorphic with the URI record with the exception of a
> > missing
> > scheme, which could be added later should the desire to use URI
> > emerge.
> >
> > It also avoids the problem of defining a URI scheme now. I think we
> > can
> > get away with this by noting that this format isn't technically a
> > URI
> > even though it closely resembles one.
>
> +1 except that I currently favor adding a flags field to indicate
> master
> KDC records.

We discussed this on the MIT developer call today and came to a
consensus. The plan looks like this:

1. Implement the following record names as TXT using exactly the same
semantics that MIT already implements (i.e. ignoring the weight
parameter):

    _kerberos-adm.REALM
    _kerberos.REALM
    _kpasswd.REALM

2. The following record names will not be implemented:

    _kerberos-master.REALM
    _krb524.REALM

3. The format of the TXT record is:

    priority:weight:flags:udp:host[:port]
    priority:weight:flags:tcp:host[:port]
    priority:weight:flags:tls:host[:port]
    priority:weight:flags:kkdcp:http://host[:port][/path]
    priority:weight:flags:kkdcp:https://host[:port][/path]

4. The flags field contains zero or more flag characters. Currently,
there is only one flag: M. The M flag indicates that this record refers
to a master KDC.

If you have any objections, please voice them now.
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Greg Hudson
On 05/31/2016 03:13 PM, Nathaniel McCallum wrote:
>     _kerberos-adm.REALM
>     _kerberos.REALM
>     _kpasswd.REALM

_kerberos.REALM TXT is currently used to look up the realm of a hostname
(see lib/krb5/os/hostrealm_dns.c), so we should use a different prefix
label, like _krb5kdc or _kdc.

I have no other objections.
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Petr Spacek
In reply to this post by Greg Hudson
On 25.5.2016 23:48, Greg Hudson wrote:

> Currently we can map from a Kerberos realm name to the TCP and/or UDP
> addresses of a KDC, kpasswd, or kadmind server.  Several of us have been
> discussing how to best extend that mechanism to include MS-KKDCP, and at
> the same time minimize the number of DNS lookups required to contact a
> realm without configuration.
>
> We had been planning to use the URI record type, but after a recent
> round of discussion, we don't think it's likely that popular DNS hosting
> providers will allow customers to create URI records (since it seems
> like no one else is using them).  Some middle-boxes would also block DNS
> queries for URI records.  That problem would be even worse if we create
> a new record type.  So, we are planning to use the TXT record type.  It
> seems unlikely that we can standardize on a TXT record through the IETF
> (except perhaps as an informational RFC), but it seems like the only
> deployable option for individuals and small organizations.
>
> If we use the TXT record type, we have to define the payload format,
> which is the primary subject of this thread.  The payload must
> communicate:
>
> * Priority, because DNS records are unordered
> * Transport type--currently one of TCP, UDP, and MS-KKDCP
> * For the TCP and UDP types, the hostname and optional port
> * For the MS-KKDCP type, the proxy URL
>
> The format must be extensible to future transport types, including
> ones that use TCP, UDP, or HTTP in different ways.  Examples could
> include Heimdal's HTTP proxy protocol or RFC 6251 (Kerberos over TLS).
>
> That leaves a lot of questions:
>
> * Do we want to include weights as well as priorities?  I think weights
>   are unnecessary complexity, even just as an optional field in the
>   format, but I seem to be in the minority.
>
> * Do we want to include master KDCs in the same query as normal KDCs, or
>   should they be in a separate record?  (Master KDCs are used as a
>   fallback by the MIT client code when we receive a KDC error which
>   could have resulted from out-of-date data due to propagation delays.)
>
>   - If we do communicate master KDCs in the same record, should it be
>     possible to exclude a master KDC from the normal server list (so
>     that it is only contacted if a normal KDC returns an error), or is
>     it sufficient to be able to assign master KDCs a low priority?
>
> * Do we want to make our payload isomorphic to the URI payload, in
>   anticipation of migrating to the URI record type in the future?  I
>   would argue that such a migration is vanishingly unlikely.  If we go
>   this way, the payload contains a priority, a weight, and a URI, so we
>   have to encode everything but the priority inside a URI, opening a
>   bunch of other inter-related questions:
>
>   - Should we register a new URI scheme to represent TCP and UDP
>     transports, or use the unregistered tcp: and udp: schemes as some
>     other applications have done?
>
>   - Should we use a new URI scheme (probably the same one as above) for
>     MS-KKDCP, or should we just use the https: URI of the proxy?
>
>   - If we do create new URI schemes, should we use use separate schemes
>     for krb5/kpasswd/kadmin, or use the same scheme since we already
>     know the application protocol going in?
>
>   - Should we use fragment identifiers (#suffix) to indicate master-ness
>     and/or the transport type, or should we use a prefix?
>
>   If we don't restrict ourselves to an isomorphic-to-URI payload format,
>   we still have to decide whether and how to represent master KDC
>   entries, but the other concerns largely go away.
>
> * At the bikeshed level, what should we use as a separator between
>   fields?  Is it more convenient for DNS administrators if we avoid
>   using whitespace as a separator?

For the record, opinions of DNS gurus from dnsop list can be found in dnsop
archives:
http://www.ietf.org/mail-archive/web/dnsop/current/msg17526.html

Message
http://www.ietf.org/mail-archive/web/dnsop/current/msg17527.html
indicates that it might be possible to standardize this if you try it.

Message
http://www.ietf.org/mail-archive/web/dnsop/current/msg17534.html
argues that URI is good enough and that TXT is a bad practice.


Pick an answer which suits you the best :-)

--
Petr Spacek  @  Red Hat
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Petr Spacek
In reply to this post by Greg Hudson
On 31.5.2016 22:20, Greg Hudson wrote:

> On 05/31/2016 03:13 PM, Nathaniel McCallum wrote:
>>     _kerberos-adm.REALM
>>     _kerberos.REALM
>>     _kpasswd.REALM
>
> _kerberos.REALM TXT is currently used to look up the realm of a hostname
> (see lib/krb5/os/hostrealm_dns.c), so we should use a different prefix
> label, like _krb5kdc or _kdc.
>
> I have no other objections.

Oh wait, I just realized that custom format in TXT RR will break one important
use-case in FreeIPA.

FreeIPA v4.4 is going to have ability to tailor SRV record priorities based on
server & client's location so clients will prefer nearby servers without any
configuration on client side.

For doing so FreeIPA needs to have access to fields 'priority' and 'server
name' in the RR so server name can be mapped to location name & priority
associated with it.

In case of SRV it is easy because RR format is standardized and DNS libraries
can work with it directly.

Custom format inside TXT record will take away this simplicity and every
single system which will want to do something similar will have to implement
parser for the custom format.

For me as an implementer this is major downside of TXT approach.

--
Petr Spacek  @  Red Hat
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Matt Rogers
In reply to this post by Greg Hudson
On 05/31, Greg Hudson wrote:

> On 05/31/2016 03:13 PM, Nathaniel McCallum wrote:
> >     _kerberos-adm.REALM
> >     _kerberos.REALM
> >     _kpasswd.REALM
>
> _kerberos.REALM TXT is currently used to look up the realm of a hostname
> (see lib/krb5/os/hostrealm_dns.c), so we should use a different prefix
> label, like _krb5kdc or _kdc.
>
> I have no other objections.

The wiki page should be up to speed now. I added some additional notes
about priority and fallback behavior that were discussed in IRC. A
quick review would be appreciated.

https://k5wiki.kerberos.org/wiki/Projects/KDC_Discovery

--
Matt Rogers
Red Hat, Inc
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Greg Hudson
In reply to this post by Petr Spacek
On 06/01/2016 10:31 AM, Petr Spacek wrote:
> FreeIPA needs to have access to fields 'priority' and 'server
> name' in the RR so server name can be mapped to location name & priority
> associated with it.
>
> In case of SRV it is easy because RR format is standardized and DNS libraries
> can work with it directly.

SRV is not really an interesting comparison unless your viewpoint is
that MS-KKDCP transport discovery just shouldn't be implemented.

If we used URI you would have easy access to the weight (assuming your
DNS library implements the URI RR type), but not to the server name,
since we would be using a Kerberos-specific URI scheme.

> Custom format inside TXT record will take away this simplicity and every
> single system which will want to do something similar will have to implement
> parser for the custom format.
>
> For me as an implementer this is major downside of TXT approach.

Sure.  Structure is good for consumers who want to know about it,
especially if they can delegate the understanding of that structure to a
library.  Structure is bad for producers who want to know as little as
possible, or for getting through middle-boxes which habitually reject
structure tags they don't understand.  It's a trade-off.
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: Kerberos transport DNS record design

Greg Hudson
In reply to this post by Matt Rogers
On 06/01/2016 10:49 AM, Matt Rogers wrote:
> The wiki page should be up to speed now. I added some additional notes
> about priority and fallback behavior that were discussed in IRC. A
> quick review would be appreciated.
>
> https://k5wiki.kerberos.org/wiki/Projects/KDC_Discovery

* I'm not sure the term "discovery" is correct, since we know a realm
name and are just trying to figure out how to contact it.  (Compare to
"discover the printers available on my local network.")

* The TXT payload is not formatted using a URI.  We believe we could
transition to URI by creating a new URI scheme and stuffing everything
after the weight into the residual part of the URI.  But there is no URI
scheme in the payload, and we don't plan to register this URI scheme
until we need it.

* The 'M' (master) flag is only relevant to KDC lookups, but other
future flags might not be.  So saying that the flags field is ignored
for kadmin and kpasswd lookups is problematic.

* I wouldn't bother describing the "tls" transport.  It was just an
example of a possible future transport.

* There is no currently defined or implemented http transport for
MS-KKDCP (there is only https).

* I would just specify that priority and weight are as defined in RFC
2782, and that weight may or may not be implemented while priority must be.

Also, we should explicitly decide whether flag letters are
case-sensitive.  In a side conversation on IRC, Simo argued that DNS
data is traditionally case-insensitive.  I don't have an opinion.
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
12