krb5 1.13 Replication enhancements

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

krb5 1.13 Replication enhancements

Richard Basch
I have uploaded my commits to github (I am still testing, but I am
reasonably in my git commit)

See https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements

 

Basically, this uplifts the 1.13-beta1 code base to the same level as I
previously provided as an enhancement for 1.12, specifically:

- Do not require the DB first_sno to be present in the ulog (this avoids
extraneous db resyncs to downstream slaves)

- Implement downstream slave notification when ulog is updated.

 

The following are my GIT commits, but it may be better to reference the
commits via the wiki in case I have to apply another fix and rebase.

 

Allow kdb_first_* not to be present in the ulog

https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d52
1c      

 

Notify downstream slaves of pending ulog updates

https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c7
63      

 

 

-----Original Message-----

From: Richard Basch [ <mailto:[hidden email]> mailto:[hidden email]]

Sent: Tuesday, September 30, 2014 3:26 PM

To: 'Greg Hudson'; '[hidden email]'

Cc: '[hidden email]'

Subject: RE: krb5-1.13-beta1 iprop

 

Yeah, re #2... I originally went through all the code which I could find
which pertained to the ulog parsing... the only other change I had was in
kproplog at the time so that it wouldn't overrun a missing entry. I had
added some defensive checks in my original code to only allow for 1 missing
entry, but upon a ulog wrap, it would reset the number.

 

I'll try to re-incorporate all of this...

 

 

-----Original Message-----

From: Greg Hudson [ <mailto:[hidden email]> mailto:[hidden email]]

Sent: Tuesday, September 30, 2014 2:58 PM

To: Richard Basch;  <mailto:[hidden email]> [hidden email]

Cc:  <mailto:[hidden email]> [hidden email]

Subject: Re: krb5-1.13-beta1 iprop

 

On 09/30/2014 01:30 PM, Richard Basch wrote:

> If you want, I believe I can have patches for 1.13 beta in the next

> couple days and publish them to github. In essence, there should be no

> changes to command-line arguments at this point, since you have iprop

> tree-based propagation already available.

>

> Will that help?

 

Yes, that will provide a better starting point.

 

> Preliminary testing also suggests that it may be causing an extraneous

> full resync (I had this problem in one of my early implementations

> too). I suspect you are resetting the first_sno when you get your

> first ulog entry (my 1.12 patches avoided resetting the first_sno

> until it wrapped so that the first "dump/restore" would not be

> followed by a second one if there were no additional ulog entries to

> apply).  I am not certain if this is what is going on, but some

> preliminary testing suggested such (again, I haven't looked at the

> 1.13

code yet).

 

See item #2 in

 
<http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problem
s>
http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems

 

Your implementation had a workaround for this issue, but it broke (at

least) the detection of ulog size changes.  I ran out of time to design a
better workaround.

 

_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

RE: krb5 1.13 Replication enhancements

Richard Basch
Revised commits (in particular, the second one only differs in the syslog
output generated, not in the core functionality). I was concerned about
excessive logging by kadmind's servicing lots of slaves, so instead of
indicating each slave being notified (which was useful during debugging),
merely indicate the number of slaves being notified (being notified does not
mean they "understand" the notification or will check-in).

 

https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d52
1c      first_sno fix

https://github.com/rbasch/krb5/commit/1bf5b212392ff2679250fcb3d3f197fee3ced3
e1      notify slaves

 

I have also documented an alternate approach to implementing the feature in
my Wiki, but it is far too involved for me to pursue. MIT should decide if
they want to keep the separation of programs as it stands today, in which
case this patch is probably among the better implementations possible.

 

 

From: Richard Basch [mailto:[hidden email]]
Sent: Wednesday, October 1, 2014 9:22 PM
To: 'Greg Hudson'; '[hidden email]'; '[hidden email]'
Cc: '[hidden email]'; 'Richard Basch'
Subject: krb5 1.13 Replication enhancements

 

I have uploaded my commits to github (I am still testing, but I am
reasonably in my git commit)

See https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements

 

Basically, this uplifts the 1.13-beta1 code base to the same level as I
previously provided as an enhancement for 1.12, specifically:

- Do not require the DB first_sno to be present in the ulog (this avoids
extraneous db resyncs to downstream slaves)

- Implement downstream slave notification when ulog is updated.

 

The following are my GIT commits, but it may be better to reference the
commits via the wiki in case I have to apply another fix and rebase.

 

Allow kdb_first_* not to be present in the ulog

https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d52
1c      

 

Notify downstream slaves of pending ulog updates

https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c7
63      

 

 

-----Original Message-----

From: Richard Basch [ <mailto:[hidden email]> mailto:[hidden email]]

Sent: Tuesday, September 30, 2014 3:26 PM

To: 'Greg Hudson'; '[hidden email]'

Cc: '[hidden email]'

Subject: RE: krb5-1.13-beta1 iprop

 

Yeah, re #2... I originally went through all the code which I could find
which pertained to the ulog parsing... the only other change I had was in
kproplog at the time so that it wouldn't overrun a missing entry. I had
added some defensive checks in my original code to only allow for 1 missing
entry, but upon a ulog wrap, it would reset the number.

 

I'll try to re-incorporate all of this...

 

 

-----Original Message-----

From: Greg Hudson [ <mailto:[hidden email]> mailto:[hidden email]]

Sent: Tuesday, September 30, 2014 2:58 PM

To: Richard Basch;  <mailto:[hidden email]> [hidden email]

Cc:  <mailto:[hidden email]> [hidden email]

Subject: Re: krb5-1.13-beta1 iprop

 

On 09/30/2014 01:30 PM, Richard Basch wrote:

> If you want, I believe I can have patches for 1.13 beta in the next

> couple days and publish them to github. In essence, there should be no

> changes to command-line arguments at this point, since you have iprop

> tree-based propagation already available.

>

> Will that help?

 

Yes, that will provide a better starting point.

 

> Preliminary testing also suggests that it may be causing an extraneous

> full resync (I had this problem in one of my early implementations

> too). I suspect you are resetting the first_sno when you get your

> first ulog entry (my 1.12 patches avoided resetting the first_sno

> until it wrapped so that the first "dump/restore" would not be

> followed by a second one if there were no additional ulog entries to

> apply).  I am not certain if this is what is going on, but some

> preliminary testing suggested such (again, I haven't looked at the

> 1.13

code yet).

 

See item #2 in

 
<http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problem
s>
http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems

 

Your implementation had a workaround for this issue, but it broke (at

least) the detection of ulog size changes.  I ran out of time to design a
better workaround.

 

_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

RE: krb5 1.13 Replication enhancements

Richard Basch
In reply to this post by Richard Basch
I'll send a revised patch later today. I introduced a problem into kproplog... it affects the first patch.


Richard Basch

(Sent via my Samsung GALAXY S™4)

-------- Original message --------
From: Richard Basch <[hidden email]>
Date: 10/02/2014  20:24  (GMT-05:00)
To: [hidden email],Greg Hudson <[hidden email]>,Tom Yu <[hidden email]>,[hidden email]
Cc: [hidden email]
Subject: RE: krb5 1.13 Replication enhancements
 
Revised commits (in particular, the second one only differs in the syslog output generated, not in the core functionality). I was concerned about excessive logging by kadmind’s servicing lots of slaves, so instead of indicating each slave being notified (which was useful during debugging), merely indicate the number of slaves being notified (being notified does not mean they “understand” the notification or will check-in).
 
https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c      first_sno fix
https://github.com/rbasch/krb5/commit/1bf5b212392ff2679250fcb3d3f197fee3ced3e1      notify slaves
 
I have also documented an alternate approach to implementing the feature in my Wiki, but it is far too involved for me to pursue. MIT should decide if they want to keep the separation of programs as it stands today, in which case this patch is probably among the better implementations possible.
 
 
From: Richard Basch [mailto:[hidden email]]
Sent: Wednesday, October 1, 2014 9:22 PM
To: 'Greg Hudson'; '[hidden email]'; '[hidden email]'
Cc: '[hidden email]'; 'Richard Basch'
Subject: krb5 1.13 Replication enhancements
 
I have uploaded my commits to github (I am still testing, but I am reasonably in my git commit)
See https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements
 
Basically, this uplifts the 1.13-beta1 code base to the same level as I previously provided as an enhancement for 1.12, specifically:
- Do not require the DB first_sno to be present in the ulog (this avoids extraneous db resyncs to downstream slaves)
- Implement downstream slave notification when ulog is updated.
 
The following are my GIT commits, but it may be better to reference the commits via the wiki in case I have to apply another fix and rebase.
 
Allow kdb_first_* not to be present in the ulog
https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d521c     
 
Notify downstream slaves of pending ulog updates
https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c763     
 
 
-----Original Message-----
From: Richard Basch [mailto:[hidden email]]
Sent: Tuesday, September 30, 2014 3:26 PM
To: 'Greg Hudson'; '[hidden email]'
Cc: '[hidden email]'
Subject: RE: krb5-1.13-beta1 iprop
 
Yeah, re #2... I originally went through all the code which I could find which pertained to the ulog parsing... the only other change I had was in kproplog at the time so that it wouldn't overrun a missing entry. I had added some defensive checks in my original code to only allow for 1 missing entry, but upon a ulog wrap, it would reset the number.
 
I'll try to re-incorporate all of this...
 
 
-----Original Message-----
From: Greg Hudson [mailto:[hidden email]]
Sent: Tuesday, September 30, 2014 2:58 PM
To: Richard Basch; [hidden email]
Cc: [hidden email]
Subject: Re: krb5-1.13-beta1 iprop
 
On 09/30/2014 01:30 PM, Richard Basch wrote:
> If you want, I believe I can have patches for 1.13 beta in the next
> couple days and publish them to github. In essence, there should be no
> changes to command-line arguments at this point, since you have iprop
> tree-based propagation already available.
>
> Will that help?
 
Yes, that will provide a better starting point.
 
> Preliminary testing also suggests that it may be causing an extraneous
> full resync (I had this problem in one of my early implementations
> too). I suspect you are resetting the first_sno when you get your
> first ulog entry (my 1.12 patches avoided resetting the first_sno
> until it wrapped so that the first "dump/restore" would not be
> followed by a second one if there were no additional ulog entries to
> apply).  I am not certain if this is what is going on, but some
> preliminary testing suggested such (again, I haven't looked at the
> 1.13
code yet).
 
See item #2 in
http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems
 
Your implementation had a workaround for this issue, but it broke (at
least) the detection of ulog size changes.  I ran out of time to design a better workaround.
 
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

RE: krb5 1.13 Replication enhancements

Richard Basch
In reply to this post by Richard Basch
I found a problem in kproplog pursuant to my "first_sno fix".

 

Revised wiki:
https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements

 

Revised commits (pursuant to "git rebase"):

https://github.com/rbasch/krb5/commit/d3189af4fc65569881d6f553d8ab34f4357946
35  first_sno fix

https://github.com/rbasch/krb5/commit/4fbe0f4b8f5aa771bc7173725f9303a3a5b627
11  notify slaves

 

 

 

From: Richard Basch [mailto:[hidden email]]
Sent: Thursday, October 2, 2014 8:25 PM
To: 'Richard Basch'; 'Greg Hudson'; '[hidden email]'; '[hidden email]'
Cc: '[hidden email]'
Subject: RE: krb5 1.13 Replication enhancements

 

Revised commits (in particular, the second one only differs in the syslog
output generated, not in the core functionality). I was concerned about
excessive logging by kadmind's servicing lots of slaves, so instead of
indicating each slave being notified (which was useful during debugging),
merely indicate the number of slaves being notified (being notified does not
mean they "understand" the notification or will check-in).

 

https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d52
1c      first_sno fix

https://github.com/rbasch/krb5/commit/1bf5b212392ff2679250fcb3d3f197fee3ced3
e1      notify slaves

 

I have also documented an alternate approach to implementing the feature in
my Wiki, but it is far too involved for me to pursue. MIT should decide if
they want to keep the separation of programs as it stands today, in which
case this patch is probably among the better implementations possible.

 

 

From: Richard Basch [mailto:[hidden email]]
Sent: Wednesday, October 1, 2014 9:22 PM
To: 'Greg Hudson'; '[hidden email]'; '[hidden email]'
Cc: '[hidden email]'; 'Richard Basch'
Subject: krb5 1.13 Replication enhancements

 

I have uploaded my commits to github (I am still testing, but I am
reasonably in my git commit)

See https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements

 

Basically, this uplifts the 1.13-beta1 code base to the same level as I
previously provided as an enhancement for 1.12, specifically:

- Do not require the DB first_sno to be present in the ulog (this avoids
extraneous db resyncs to downstream slaves)

- Implement downstream slave notification when ulog is updated.

 

The following are my GIT commits, but it may be better to reference the
commits via the wiki in case I have to apply another fix and rebase.

 

Allow kdb_first_* not to be present in the ulog

https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d52
1c      

 

Notify downstream slaves of pending ulog updates

https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c7
63      

 

 

-----Original Message-----

From: Richard Basch [ <mailto:[hidden email]> mailto:[hidden email]]

Sent: Tuesday, September 30, 2014 3:26 PM

To: 'Greg Hudson'; '[hidden email]'

Cc: '[hidden email]'

Subject: RE: krb5-1.13-beta1 iprop

 

Yeah, re #2... I originally went through all the code which I could find
which pertained to the ulog parsing... the only other change I had was in
kproplog at the time so that it wouldn't overrun a missing entry. I had
added some defensive checks in my original code to only allow for 1 missing
entry, but upon a ulog wrap, it would reset the number.

 

I'll try to re-incorporate all of this...

 

 

-----Original Message-----

From: Greg Hudson [ <mailto:[hidden email]> mailto:[hidden email]]

Sent: Tuesday, September 30, 2014 2:58 PM

To: Richard Basch;  <mailto:[hidden email]> [hidden email]

Cc:  <mailto:[hidden email]> [hidden email]

Subject: Re: krb5-1.13-beta1 iprop

 

On 09/30/2014 01:30 PM, Richard Basch wrote:

> If you want, I believe I can have patches for 1.13 beta in the next

> couple days and publish them to github. In essence, there should be no

> changes to command-line arguments at this point, since you have iprop

> tree-based propagation already available.

>

> Will that help?

 

Yes, that will provide a better starting point.

 

> Preliminary testing also suggests that it may be causing an extraneous

> full resync (I had this problem in one of my early implementations

> too). I suspect you are resetting the first_sno when you get your

> first ulog entry (my 1.12 patches avoided resetting the first_sno

> until it wrapped so that the first "dump/restore" would not be

> followed by a second one if there were no additional ulog entries to

> apply).  I am not certain if this is what is going on, but some

> preliminary testing suggested such (again, I haven't looked at the

> 1.13

code yet).

 

See item #2 in

 
<http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problem
s>
http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems

 

Your implementation had a workaround for this issue, but it broke (at

least) the detection of ulog size changes.  I ran out of time to design a
better workaround.

 

_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

RE: krb5 1.13 Replication enhancements

Richard Basch
In reply to this post by Richard Basch
I found an issue with kdb_first_sno not being set during a DB restore which
contributed to extra resyncs. Now, I believe all my prior 1.12 contributions
are now covered by these patches.

 

Updated wiki:
https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements

 


Revised commits (pursuant to "git rebase"):

https://github.com/rbasch/krb5/commit/d9ec51d87f9fc4e1592423bf7562895fad2752
c2  first_sno fix

https://github.com/rbasch/krb5/commit/f4ed3369f2c7a0ec6d834f46677d8ffc4e2223
f4  notify slaves

 

 

 

From: Richard Basch [mailto:[hidden email]]
Sent: Wednesday, October 8, 2014 9:42 PM
To: 'Richard Basch'; 'Greg Hudson'; '[hidden email]'; '[hidden email]'
Cc: '[hidden email]'
Subject: RE: krb5 1.13 Replication enhancements

 

I found a problem in kproplog pursuant to my "first_sno fix".

 

Revised wiki:
https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements

 


Revised commits (pursuant to "git rebase"):

https://github.com/rbasch/krb5/commit/d3189af4fc65569881d6f553d8ab34f4357946
35  first_sno fix

https://github.com/rbasch/krb5/commit/4fbe0f4b8f5aa771bc7173725f9303a3a5b627
11  notify slaves

 

 

 

From: Richard Basch [mailto:[hidden email]]
Sent: Thursday, October 2, 2014 8:25 PM
To: 'Richard Basch'; 'Greg Hudson'; '[hidden email]'; '[hidden email]'
Cc: '[hidden email]'
Subject: RE: krb5 1.13 Replication enhancements

 

Revised commits (in particular, the second one only differs in the syslog
output generated, not in the core functionality). I was concerned about
excessive logging by kadmind's servicing lots of slaves, so instead of
indicating each slave being notified (which was useful during debugging),
merely indicate the number of slaves being notified (being notified does not
mean they "understand" the notification or will check-in).

 

https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d52
1c      first_sno fix

https://github.com/rbasch/krb5/commit/1bf5b212392ff2679250fcb3d3f197fee3ced3
e1      notify slaves

 

I have also documented an alternate approach to implementing the feature in
my Wiki, but it is far too involved for me to pursue. MIT should decide if
they want to keep the separation of programs as it stands today, in which
case this patch is probably among the better implementations possible.

 

 

From: Richard Basch [mailto:[hidden email]]
Sent: Wednesday, October 1, 2014 9:22 PM
To: 'Greg Hudson'; '[hidden email]'; '[hidden email]'
Cc: '[hidden email]'; 'Richard Basch'
Subject: krb5 1.13 Replication enhancements

 

I have uploaded my commits to github (I am still testing, but I am
reasonably in my git commit)

See https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements

 

Basically, this uplifts the 1.13-beta1 code base to the same level as I
previously provided as an enhancement for 1.12, specifically:

- Do not require the DB first_sno to be present in the ulog (this avoids
extraneous db resyncs to downstream slaves)

- Implement downstream slave notification when ulog is updated.

 

The following are my GIT commits, but it may be better to reference the
commits via the wiki in case I have to apply another fix and rebase.

 

Allow kdb_first_* not to be present in the ulog

https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d52
1c      

 

Notify downstream slaves of pending ulog updates

https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c7
63      

 

 

-----Original Message-----

From: Richard Basch [ <mailto:[hidden email]> mailto:[hidden email]]

Sent: Tuesday, September 30, 2014 3:26 PM

To: 'Greg Hudson'; '[hidden email]'

Cc: '[hidden email]'

Subject: RE: krb5-1.13-beta1 iprop

 

Yeah, re #2... I originally went through all the code which I could find
which pertained to the ulog parsing... the only other change I had was in
kproplog at the time so that it wouldn't overrun a missing entry. I had
added some defensive checks in my original code to only allow for 1 missing
entry, but upon a ulog wrap, it would reset the number.

 

I'll try to re-incorporate all of this...

 

 

-----Original Message-----

From: Greg Hudson [ <mailto:[hidden email]> mailto:[hidden email]]

Sent: Tuesday, September 30, 2014 2:58 PM

To: Richard Basch;  <mailto:[hidden email]> [hidden email]

Cc:  <mailto:[hidden email]> [hidden email]

Subject: Re: krb5-1.13-beta1 iprop

 

On 09/30/2014 01:30 PM, Richard Basch wrote:

> If you want, I believe I can have patches for 1.13 beta in the next

> couple days and publish them to github. In essence, there should be no

> changes to command-line arguments at this point, since you have iprop

> tree-based propagation already available.

>

> Will that help?

 

Yes, that will provide a better starting point.

 

> Preliminary testing also suggests that it may be causing an extraneous

> full resync (I had this problem in one of my early implementations

> too). I suspect you are resetting the first_sno when you get your

> first ulog entry (my 1.12 patches avoided resetting the first_sno

> until it wrapped so that the first "dump/restore" would not be

> followed by a second one if there were no additional ulog entries to

> apply).  I am not certain if this is what is going on, but some

> preliminary testing suggested such (again, I haven't looked at the

> 1.13

code yet).

 

See item #2 in

 
<http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problem
s>
http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems

 

Your implementation had a workaround for this issue, but it broke (at

least) the detection of ulog size changes.  I ran out of time to design a
better workaround.

 

_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

RE: krb5 1.13 Replication enhancements

Richard Basch
In reply to this post by Richard Basch
FYI. My "git diff" patches from beta1 applied cleanly into 1.13 "release".

No surprise - I didn't expect any code change between 1.13-beta1 and 1.13.

 

Hopefully, this can get rolled into 1.14.

 


 

From: Richard Basch [mailto:[hidden email]]
Sent: Saturday, October 11, 2014 6:48 PM
To: 'Richard Basch'; 'Greg Hudson'; '[hidden email]'; '[hidden email]'
Cc: '[hidden email]'; '[hidden email]';
'[hidden email]'
Subject: RE: krb5 1.13 Replication enhancements

 

I found an issue with kdb_first_sno not being set during a DB restore which
contributed to extra resyncs. Now, I believe all my prior 1.12 contributions
are now covered by these patches.

 

Updated wiki:
https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements

 


Revised commits (pursuant to "git rebase"):

https://github.com/rbasch/krb5/commit/d9ec51d87f9fc4e1592423bf7562895fad2752
c2  first_sno fix

https://github.com/rbasch/krb5/commit/f4ed3369f2c7a0ec6d834f46677d8ffc4e2223
f4  notify slaves

 

 

 

From: Richard Basch [mailto:[hidden email]]
Sent: Wednesday, October 8, 2014 9:42 PM
To: 'Richard Basch'; 'Greg Hudson'; '[hidden email]'; '[hidden email]'
Cc: '[hidden email]'
Subject: RE: krb5 1.13 Replication enhancements

 

I found a problem in kproplog pursuant to my "first_sno fix".

 

Revised wiki:
https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements

 


Revised commits (pursuant to "git rebase"):

https://github.com/rbasch/krb5/commit/d3189af4fc65569881d6f553d8ab34f4357946
35  first_sno fix

https://github.com/rbasch/krb5/commit/4fbe0f4b8f5aa771bc7173725f9303a3a5b627
11  notify slaves

 

 

 

From: Richard Basch [mailto:[hidden email]]
Sent: Thursday, October 2, 2014 8:25 PM
To: 'Richard Basch'; 'Greg Hudson'; '[hidden email]'; '[hidden email]'
Cc: '[hidden email]'
Subject: RE: krb5 1.13 Replication enhancements

 

Revised commits (in particular, the second one only differs in the syslog
output generated, not in the core functionality). I was concerned about
excessive logging by kadmind's servicing lots of slaves, so instead of
indicating each slave being notified (which was useful during debugging),
merely indicate the number of slaves being notified (being notified does not
mean they "understand" the notification or will check-in).

 

https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d52
1c      first_sno fix

https://github.com/rbasch/krb5/commit/1bf5b212392ff2679250fcb3d3f197fee3ced3
e1      notify slaves

 

I have also documented an alternate approach to implementing the feature in
my Wiki, but it is far too involved for me to pursue. MIT should decide if
they want to keep the separation of programs as it stands today, in which
case this patch is probably among the better implementations possible.

 

 

From: Richard Basch [mailto:[hidden email]]
Sent: Wednesday, October 1, 2014 9:22 PM
To: 'Greg Hudson'; '[hidden email]'; '[hidden email]'
Cc: '[hidden email]'; 'Richard Basch'
Subject: krb5 1.13 Replication enhancements

 

I have uploaded my commits to github (I am still testing, but I am
reasonably in my git commit)

See https://github.com/rbasch/krb5/wiki/1.13-Replication-enhancements

 

Basically, this uplifts the 1.13-beta1 code base to the same level as I
previously provided as an enhancement for 1.12, specifically:

- Do not require the DB first_sno to be present in the ulog (this avoids
extraneous db resyncs to downstream slaves)

- Implement downstream slave notification when ulog is updated.

 

The following are my GIT commits, but it may be better to reference the
commits via the wiki in case I have to apply another fix and rebase.

 

Allow kdb_first_* not to be present in the ulog

https://github.com/rbasch/krb5/commit/ce238c4a19e117caedc7ce53f4780270b86d52
1c      

 

Notify downstream slaves of pending ulog updates

https://github.com/rbasch/krb5/commit/db0c450f6c09f4623bff3c3aff16794ba4e2c7
63      

 

 

-----Original Message-----

From: Richard Basch [ <mailto:[hidden email]> mailto:[hidden email]]

Sent: Tuesday, September 30, 2014 3:26 PM

To: 'Greg Hudson'; '[hidden email]'

Cc: '[hidden email]'

Subject: RE: krb5-1.13-beta1 iprop

 

Yeah, re #2... I originally went through all the code which I could find
which pertained to the ulog parsing... the only other change I had was in
kproplog at the time so that it wouldn't overrun a missing entry. I had
added some defensive checks in my original code to only allow for 1 missing
entry, but upon a ulog wrap, it would reset the number.

 

I'll try to re-incorporate all of this...

 

 

-----Original Message-----

From: Greg Hudson [ <mailto:[hidden email]> mailto:[hidden email]]

Sent: Tuesday, September 30, 2014 2:58 PM

To: Richard Basch;  <mailto:[hidden email]> [hidden email]

Cc:  <mailto:[hidden email]> [hidden email]

Subject: Re: krb5-1.13-beta1 iprop

 

On 09/30/2014 01:30 PM, Richard Basch wrote:

> If you want, I believe I can have patches for 1.13 beta in the next

> couple days and publish them to github. In essence, there should be no

> changes to command-line arguments at this point, since you have iprop

> tree-based propagation already available.

>

> Will that help?

 

Yes, that will provide a better starting point.

 

> Preliminary testing also suggests that it may be causing an extraneous

> full resync (I had this problem in one of my early implementations

> too). I suspect you are resetting the first_sno when you get your

> first ulog entry (my 1.12 patches avoided resetting the first_sno

> until it wrapped so that the first "dump/restore" would not be

> followed by a second one if there were no additional ulog entries to

> apply).  I am not certain if this is what is going on, but some

> preliminary testing suggested such (again, I haven't looked at the

> 1.13

code yet).

 

See item #2 in

 
<http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problem
s>
http://k5wiki.kerberos.org/wiki/Projects/Hierarchical_iprop#Related_problems

 

Your implementation had a workaround for this issue, but it broke (at

least) the detection of ulog size changes.  I ran out of time to design a
better workaround.

 

_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev