C99 Features

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

C99 Features

Nathaniel McCallum-5
It has been 16 years. GCC has had support for many C99 features for a
LongTimeNowTM. Clang has had them since the beginning. Clang also now
has official Windows builds (as well as many other platforms).

Of course, MSVC still lags behind. However, they have started
implementing features, including the following since MSVC 2013 [1]:
 * _Bool
 * Compound literals
 * Designated initializers
 * Mixing declarations with code

MS has also implemented most of the C99 libraries[2] and has already
announced complete support for the C99 standard library in MSVC
2015[3]. One of the big elephants in the room is VLAs. MS is unlikely
to ever support them. But, it is also MS's stated policy to only
incorporate C features that are required for C++[4]. I'm not sure that
MIT should hold back support for new C features because MS only wants
to ship a C++ compiler.

So how about it? Can MIT start using C99 features? Even a subset of the
features would be helpful. I particularly care about all the above C99
features that MS implemented plus VLAs. The latter, in particular, can
eliminate a lot of heap allocations.

Nathaniel

[1] http://blogs.msdn.com/b/vcblog/archive/2013/06/28/c-11-14-stl
-features-fixes-and-breaking-changes-in-vs-2013.aspx
[2] http://blogs.msdn.com/b/vcblog/archive/2013/07/19/c99-library
-support-in-visual-studio-2013.aspx?PageIndex=2
[3] http://blogs.msdn.com/b/vcblog/archive/2014/11/17/c-11-14-17
-features-in-vs-2015-preview.aspx
[4] http://blogs.msdn.com/b/vcblog/archive/2015/04/29/c-11-14-17
-features-in-vs-2015-rc.aspx


_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Greg Hudson
On 06/15/2015 09:06 PM, Nathaniel McCallum wrote:
> So how about it? Can MIT start using C99 features?

There are four questions here:

1. Can we reasonably require MSVC 2013 for the Windows build?  I don't
know of a compelling reason why we can't, but there may be reasons I
don't know about.

2. Can we abandon MSVC for the Windows build in favor of mingw or clang,
in order to get VLA support?  I think the practical answer here is no,
in that we don't expect to commit the resources to investigate this
possibility in the near future.

3. Should we allow MSVC-unsupported C99 features in code we don't
currently build on Windows?  (This includes the KDC, KDB library,
PKINIT, and everything related to kadmin including gssrpc.)  I don't
really feel strongly either way, but it doesn't buy us a lot to allow it
in just a few places, and the inconsistency could be confusing.

4. Are there any C99 features we don't want to use, because they don't
mesh with our BSD KNF-inspired style or for other reasons?  Going over
the specific features you mentioned:

* Designated initializers and compound literals don't seem to present
any stylistic or practical issues.  VLAs also don't seem to present
issues except for the lack of MSVC support.

* Does _Bool present any issues if it creeps into public ABIs?  I found
http://yarchive.net/comp/linux/bool.html but it's from 2009, and might
be specific to the kernel.  It might also just be confusing to have
"krb5_boolean" and "bool" in the same code base when they aren't
generally the same size (krb5_boolean is a typedef for unsigned int).

* Our style guide discourages declaring variables in interior scope.
The justification (which predates my involvement) has to do with
debugging convenience and limiting function complexity; I personally
find that code is a little easier to read if it doesn't have type
declarations mixed in with statements.  If there are good reasons to
avoid interior scope declarations, those reasons may also apply to mixed
declarations and code.
http://k5wiki.kerberos.org/wiki/Coding_style/Practices#Local_variables

_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Chris Hecker
I still build on vs2008, for what it's worth.

Chris
 On Jun 15, 2015 10:24 PM, "Greg Hudson" <[hidden email]> wrote:

> On 06/15/2015 09:06 PM, Nathaniel McCallum wrote:
> > So how about it? Can MIT start using C99 features?
>
> There are four questions here:
>
> 1. Can we reasonably require MSVC 2013 for the Windows build?  I don't
> know of a compelling reason why we can't, but there may be reasons I
> don't know about.
>
> 2. Can we abandon MSVC for the Windows build in favor of mingw or clang,
> in order to get VLA support?  I think the practical answer here is no,
> in that we don't expect to commit the resources to investigate this
> possibility in the near future.
>
> 3. Should we allow MSVC-unsupported C99 features in code we don't
> currently build on Windows?  (This includes the KDC, KDB library,
> PKINIT, and everything related to kadmin including gssrpc.)  I don't
> really feel strongly either way, but it doesn't buy us a lot to allow it
> in just a few places, and the inconsistency could be confusing.
>
> 4. Are there any C99 features we don't want to use, because they don't
> mesh with our BSD KNF-inspired style or for other reasons?  Going over
> the specific features you mentioned:
>
> * Designated initializers and compound literals don't seem to present
> any stylistic or practical issues.  VLAs also don't seem to present
> issues except for the lack of MSVC support.
>
> * Does _Bool present any issues if it creeps into public ABIs?  I found
> http://yarchive.net/comp/linux/bool.html but it's from 2009, and might
> be specific to the kernel.  It might also just be confusing to have
> "krb5_boolean" and "bool" in the same code base when they aren't
> generally the same size (krb5_boolean is a typedef for unsigned int).
>
> * Our style guide discourages declaring variables in interior scope.
> The justification (which predates my involvement) has to do with
> debugging convenience and limiting function complexity; I personally
> find that code is a little easier to read if it doesn't have type
> declarations mixed in with statements.  If there are good reasons to
> avoid interior scope declarations, those reasons may also apply to mixed
> declarations and code.
> http://k5wiki.kerberos.org/wiki/Coding_style/Practices#Local_variables
>
> _______________________________________________
> krbdev mailing list             [hidden email]
> https://mailman.mit.edu/mailman/listinfo/krbdev
>
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Nathaniel McCallum-5
In reply to this post by Greg Hudson
On Tue, 2015-06-16 at 01:20 -0400, Greg Hudson wrote:
> On 06/15/2015 09:06 PM, Nathaniel McCallum wrote:
> > So how about it? Can MIT start using C99 features?
>
> There are four questions here:
>
> 1. Can we reasonably require MSVC 2013 for the Windows build?  I
> don't
> know of a compelling reason why we can't, but there may be reasons I
> don't know about.

MSVC 2015 will be released soon. MSVC 2013 seems reasonable given that
users who don't wish to use it have a free (freedom and beer)
alternative: clang.

> 2. Can we abandon MSVC for the Windows build in favor of mingw or
> clang,
> in order to get VLA support?  I think the practical answer here is
> no,
> in that we don't expect to commit the resources to investigate this
> possibility in the near future.

I don't think MIT should do this. I think MIT should require MSVC 2013.
Users who can't use MSVC 2013 should investigate mingw/clang.

> 3. Should we allow MSVC-unsupported C99 features in code we don't
> currently build on Windows?  (This includes the KDC, KDB library,
> PKINIT, and everything related to kadmin including gssrpc.)  I don't
> really feel strongly either way, but it doesn't buy us a lot to allow
> it
> in just a few places, and the inconsistency could be confusing.

I think VLAs buys a lot.

> 4. Are there any C99 features we don't want to use, because they
> don't
> mesh with our BSD KNF-inspired style or for other reasons?  Going
> over
> the specific features you mentioned:
>
> * Designated initializers and compound literals don't seem to present
> any stylistic or practical issues.

Agreed. I think compound literals help a lot with data type shuffling.

>  * VLAs also don't seem to present
> issues except for the lack of MSVC support.

Of all the features, I find VLAs most valuable.

https://github.com/npmccallum/krb5/commit/9345f2b92712d39573a17b75fcae8
2bfb0f5009e

> * Does _Bool present any issues if it creeps into public ABIs?  I
> found
> http://yarchive.net/comp/linux/bool.html but it's from 2009, and
> might
> be specific to the kernel.  It might also just be confusing to have
> "krb5_boolean" and "bool" in the same code base when they aren't
> generally the same size (krb5_boolean is a typedef for unsigned int).

I don't care either way on this one.

> * Our style guide discourages declaring variables in interior scope.
> The justification (which predates my involvement) has to do with
> debugging convenience and limiting function complexity; I personally
> find that code is a little easier to read if it doesn't have type
> declarations mixed in with statements.  If there are good reasons to
> avoid interior scope declarations, those reasons may also apply to
> mixed
> declarations and code.
> http://k5wiki.kerberos.org/wiki/Coding_style/Practices#Local_variable
> s

My personal preference here is the following.

Declare variables at the beginning of its appropriate scope. For
example:
context ctx;
for (int i = 0; i < x; i++) {
  foo *x;

  x = make_foo(&ctx);
  assign_foo(x);
}

Allow mixing of statements only in initializers at the start of scope:
krb5_data x = { 0, strlen(str), str };

... or VLAs:
len = get_buffer_len();
char buffer[len];

Other than that, I don't mix declarations and statements.

Nathaniel
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Nathaniel McCallum-5
In reply to this post by Chris Hecker
Is clang an option for you?

On Tue, 2015-06-16 at 01:54 -0700, Chris Hecker wrote:

> I still build on vs2008, for what it's worth.
> Chris
> On Jun 15, 2015 10:24 PM, "Greg Hudson" <[hidden email]> wrote:
> > On 06/15/2015 09:06 PM, Nathaniel McCallum wrote:
> > > So how about it? Can MIT start using C99 features?
> >
> > There are four questions here:
> >
> > 1. Can we reasonably require MSVC 2013 for the Windows build?  I
> > don't
> > know of a compelling reason why we can't, but there may be reasons
> > I
> > don't know about.
> >
> > 2. Can we abandon MSVC for the Windows build in favor of mingw or
> > clang,
> > in order to get VLA support?  I think the practical answer here is
> > no,
> > in that we don't expect to commit the resources to investigate this
> > possibility in the near future.
> >
> > 3. Should we allow MSVC-unsupported C99 features in code we don't
> > currently build on Windows?  (This includes the KDC, KDB library,
> > PKINIT, and everything related to kadmin including gssrpc.)  I
> > don't
> > really feel strongly either way, but it doesn't buy us a lot to
> > allow it
> > in just a few places, and the inconsistency could be confusing.
> >
> > 4. Are there any C99 features we don't want to use, because they
> > don't
> > mesh with our BSD KNF-inspired style or for other reasons?  Going
> > over
> > the specific features you mentioned:
> >
> > * Designated initializers and compound literals don't seem to
> > present
> > any stylistic or practical issues.  VLAs also don't seem to present
> > issues except for the lack of MSVC support.
> >
> > * Does _Bool present any issues if it creeps into public ABIs?  I
> > found
> > http://yarchive.net/comp/linux/bool.html but it's from 2009, and
> > might
> > be specific to the kernel.  It might also just be confusing to have
> > "krb5_boolean" and "bool" in the same code base when they aren't
> > generally the same size (krb5_boolean is a typedef for unsigned
> > int).
> >
> > * Our style guide discourages declaring variables in interior
> > scope.
> > The justification (which predates my involvement) has to do with
> > debugging convenience and limiting function complexity; I
> > personally
> > find that code is a little easier to read if it doesn't have type
> > declarations mixed in with statements.  If there are good reasons
> > to
> > avoid interior scope declarations, those reasons may also apply to
> > mixed
> > declarations and code.
> > http://k5wiki.kerberos.org/wiki/Coding_style/Practices#Local_variab
> > les
> >
> > _______________________________________________
> > krbdev mailing list             [hidden email]
> > https://mailman.mit.edu/mailman/listinfo/krbdev
> >
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Theodore Ts'o-2
In reply to this post by Greg Hudson
On Tue, Jun 16, 2015 at 01:20:51AM -0400, Greg Hudson wrote:
> * Our style guide discourages declaring variables in interior scope.
> The justification (which predates my involvement) has to do with
> debugging convenience and limiting function complexity; I personally
> find that code is a little easier to read if it doesn't have type
> declarations mixed in with statements.  If there are good reasons to
> avoid interior scope declarations, those reasons may also apply to mixed
> declarations and code.
> http://k5wiki.kerberos.org/wiki/Coding_style/Practices#Local_variables

This one is my fault, and dates back to a time when gdb would
completely fail to allow you to examine at variables that were not
declared at the beginning of a function.  This made debugging problems
to be quite painful, to say the least.  This has been fixed with
modern versions of gdb, so the original reason why I had promulgated
that rule was long gone.

Limiting function complexity is still a good reason, which is why some
coding practices will use an 8 space tab stop and 80 column line
width, and point out that if you're indenting more deeply than three
or four tab stops, you should seriously rethink whether the function
should be refactored.  But as a reason for itself, I don't think it's
strong enough to support such a rule, since as Greg points out, there
are some good reasons for declaring variables inside a narrower scope.

Looking at the Kerberos Wiki (which postdates my involvement), I see
that it now points out that the gdb restriction might still be the
case on Solaris.  I'm not sure whether or not that is true, since
blessedly my responsibilities these days allow me to pretend that
Solaris doesn't exist and doesn't matter.  :-)

In any case, it's up to those whose hands who now have stewardship for
Kerberos to decide whether or not any particular part of the style
guide still makes sense, but I thought adding a bit of historical
color might be helpful.

Cheers,

                                                - Ted
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

RE: C99 Features

Danilo Almeida-2
In reply to this post by Nathaniel McCallum-5
Nathaniel wrote:
> I think VLAs buys a lot.

It seems that VLA could introduce new issues or invariants since you can no longer check for allocation failure.

- Danilo


_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Nathaniel McCallum-5
On Wed, 2015-06-17 at 00:02 +0000, Danilo Almeida wrote:
> Nathaniel wrote:
> > I think VLAs buys a lot.
>
> It seems that VLA could introduce new issues or invariants since you
> can no longer check for allocation failure.

You can't check for allocation failure on most operating systems' heap
allocation these days. Now, the failure in the heap vs stack cases is
somewhat different; and you have to be cautious to avoid some security
issues. But with careful use, there is a lot of benefit.

Nathaniel
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Greg Hudson
On 06/17/2015 02:32 PM, Nathaniel McCallum wrote:
> You can't check for allocation failure on most operating systems' heap
> allocation these days.

MIT krb5 does not assume that it is operating in such an environment, or
we could simplify an awful lot of code.  I've gently brought this up
once or twice and the consensus has been that we should continue trying
to rigorously handle malloc failures, even if not all environments make
such checks useful.

A more nuanced argument is that you can't check for the success of stack
allocation even when allocating fixed-sized automatic variables.  When
an attacker can control stack use without reasonable bounds, that
creates a DOS vector, whether the stack overrun is due to recursion or
VLAs.  The same is true for heap use without reasonable bound, although
in a threaded environment the reasonable bound for stack use has to be
significantly lower than for heap use.

In a threaded environment, VLAs of attacker-controlled size, which are
accessed in non-sequential order, could cause more significant security
issues than a DOS; you might skip over the guard page and overwrite a
different thread's stack.  The same is theoretically possible with
fixed-sized automatic variables, but there would have to be a lot of
them (or a few very large ones).

> But with careful use, there is a lot of benefit.

I agree with the conclusion.

_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Nathaniel McCallum-5
On Wed, 2015-06-17 at 14:46 -0400, Greg Hudson wrote:

> On 06/17/2015 02:32 PM, Nathaniel McCallum wrote:
> > You can't check for allocation failure on most operating systems'
> > heap
> > allocation these days.
>
> MIT krb5 does not assume that it is operating in such an environment,
> or
> we could simplify an awful lot of code.  I've gently brought this up
> once or twice and the consensus has been that we should continue
> trying
> to rigorously handle malloc failures, even if not all environments
> make
> such checks useful.

FTR, I was not suggesting we should ignore failures. We shouldn't. I
only meant that on many platforms such handling is not likely to catch
an actual OOM condition.

> A more nuanced argument is that you can't check for the success of
> stack
> allocation even when allocating fixed-sized automatic variables.
>  When
> an attacker can control stack use without reasonable bounds, that
> creates a DOS vector, whether the stack overrun is due to recursion
> or
> VLAs.  The same is true for heap use without reasonable bound,
> although
> in a threaded environment the reasonable bound for stack use has to
> be
> significantly lower than for heap use.
>
> In a threaded environment, VLAs of attacker-controlled size, which
> are
> accessed in non-sequential order, could cause more significant
> security
> issues than a DOS; you might skip over the guard page and overwrite a
> different thread's stack.  The same is theoretically possible with
> fixed-sized automatic variables, but there would have to be a lot of
> them (or a few very large ones).
>
> > But with careful use, there is a lot of benefit.
>
> I agree with the conclusion.
>
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Nico Williams
In reply to this post by Nathaniel McCallum-5
On Wed, Jun 17, 2015 at 02:32:15PM -0400, Nathaniel McCallum wrote:

> On Wed, 2015-06-17 at 00:02 +0000, Danilo Almeida wrote:
> > Nathaniel wrote:
> > > I think VLAs buys a lot.
> >
> > It seems that VLA could introduce new issues or invariants since you
> > can no longer check for allocation failure.
>
> You can't check for allocation failure on most operating systems' heap
> allocation these days. Now, the failure in the heap vs stack cases is
> somewhat different; and you have to be cautious to avoid some security
> issues. But with careful use, there is a lot of benefit.

Heap allocation failures are rather different than alloca() or VLA
allocation failures.  alloca(), for example, has undefined behavior on
failure for obvious reasons: sp + size might land in a mapped page for a
different mapping, going well past the stack's guard page.

In practice, if the stack grows down and the elements of the VLA are
accessed from higher to lower (or the reverse if the stack grows up)
then the guard page should still work, but this isn't defined behavior.

Both are fine when the sizes are naturally limited to small sizes, but
this requires more review effort.  Alternatively one could have a macro
to guard against unsafe array sizes.  I'd rather VLAs and alloca() were
frowned upon (though not forbidden).

Nico
--
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Nico Williams
In reply to this post by Nathaniel McCallum-5
On Wed, Jun 17, 2015 at 03:27:40PM -0400, Nathaniel McCallum wrote:
> FTR, I was not suggesting we should ignore failures. We shouldn't. I
> only meant that on many platforms such handling is not likely to catch
> an actual OOM condition.

A segfault ia very different from "undefined behavior"...

I suppose a C run-time could check that VLA and alloca() allocations
don't blow past hard limits on stack size, but I suspect none does: an
ABI would have to be defined for performing that check, and doing so no
slower than a thread-local access.

It sure would be nice if such an ABI existed, and the run-times used it.
But it couldn't really be required for implementations.

Nico
--
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

RE: C99 Features

Danilo Almeida-2
In reply to this post by Nico Williams
<quote from="Nico">
Both are fine when the sizes are naturally limited to small sizes, but this requires more review effort.  Alternatively one could have a macro to guard against unsafe array sizes.  I'd rather VLAs and alloca() were frowned upon (though not forbidden).
</quote>

I am not sure how such a macro would work.

Otherwise, I mostly agree with Nico, except that I am not comfortable with the "though not forbidden" part.  Nathaniel, do you have any performance numbers would help the case for the extra effort (and potential risk)?

- Danilo


_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Nathaniel McCallum-5
On Thu, 2015-06-18 at 01:13 +0000, Danilo Almeida wrote:

> <quote from="Nico">
> Both are fine when the sizes are naturally limited to small sizes,
> but this requires more review effort.  Alternatively one could have a
> macro to guard against unsafe array sizes.  I'd rather VLAs and
> alloca() were frowned upon (though not forbidden).
> </quote>
>
> I am not sure how such a macro would work.
>
> Otherwise, I mostly agree with Nico, except that I am not comfortable
> with the "though not forbidden" part.  Nathaniel, do you have any
> performance numbers would help the case for the extra effort (and
> potential risk)?

Well, take common inner-function heap allocations and turn them into
stack allocations. That is a significant performance gain.

VLAs make perfect sense for operations like fx_cf2 where you need to
mix two keys together using intermediary buffers. If you validate that
the input keys are valid types then you know that the buffer for that
type will be a sensible size to put on the stack. This eliminates two
unnecessary heap allocations.

Nathaniel
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Nathaniel McCallum-5
In reply to this post by Nico Williams
On Wed, 2015-06-17 at 18:06 -0500, Nico Williams wrote:

> On Wed, Jun 17, 2015 at 02:32:15PM -0400, Nathaniel McCallum wrote:
> > On Wed, 2015-06-17 at 00:02 +0000, Danilo Almeida wrote:
> > > Nathaniel wrote:
> > > > I think VLAs buys a lot.
> > >
> > > It seems that VLA could introduce new issues or invariants since
> > > you
> > > can no longer check for allocation failure.
> >
> > You can't check for allocation failure on most operating systems'
> > heap
> > allocation these days. Now, the failure in the heap vs stack cases
> > is
> > somewhat different; and you have to be cautious to avoid some
> > security
> > issues. But with careful use, there is a lot of benefit.
>
> Heap allocation failures are rather different than alloca() or VLA
> allocation failures.  alloca(), for example, has undefined behavior
> on
> failure for obvious reasons: sp + size might land in a mapped page
> for a
> different mapping, going well past the stack's guard page.
>
> In practice, if the stack grows down and the elements of the VLA are
> accessed from higher to lower (or the reverse if the stack grows up)
> then the guard page should still work, but this isn't defined
> behavior.
>
> Both are fine when the sizes are naturally limited to small sizes,
> but
> this requires more review effort.  Alternatively one could have a
> macro
> to guard against unsafe array sizes.  I'd rather VLAs and alloca()
> were
> frowned upon (though not forbidden).

Most of this is greatly diminished by the various stack protection
techniques found in modern compilers. It is getting harder and harder
to even trigger these kinds of errors.

Nathaniel
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Nico Williams
In reply to this post by Danilo Almeida-2
On Thu, Jun 18, 2015 at 01:13:40AM +0000, Danilo Almeida wrote:
> <quote from="Nico">
> Both are fine when the sizes are naturally limited to small sizes, but
> this requires more review effort.  Alternatively one could have a
> macro to guard against unsafe array sizes.  I'd rather VLAs and
> alloca() were frowned upon (though not forbidden).
> </quote>
>
> I am not sure how such a macro would work.

A macro could work for alloca:

void *alloca_abort() { abort(); }

#define GUARDED_ALLOCA(requested, max) \
    (requested <= max ? alloca(requested) : alloca_abort())

Obviously there may be other failure handling options.

Nico
--
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Nico Williams
In reply to this post by Nathaniel McCallum-5
On Thu, Jun 18, 2015 at 12:29:39AM -0400, Nathaniel McCallum wrote:
> Most of this is greatly diminished by the various stack protection
> techniques found in modern compilers. It is getting harder and harder
> to even trigger these kinds of errors.

Standard techniques for protecting against buffer overflows don't help
here.  The problem is that the emitted code cannot know where the end of
the stack lies.  What can the compiler do?  It could:

 - Write a zero to one word of each page of the array in sequence,
   hoping that the guard page will never be missed and will cause stack
   growth or SEGFAULT in all cases.

   This requires knowing the page size for the stack, but assuming a
   small page seems reasonable.

   This can also be done in user code, but it does require knowing the
   direction in which the stack grows.

   (Presumably the performance impact of this probing should be minimal.)

 - Call a system call to ensure the stack is large enough for the
   requested allocation.

   This requires adding that system call to the ABI.

Whatever the compiler does, the result has to be free of race
conditions.

Do compilers emit such code?
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Nico Williams
In reply to this post by Nathaniel McCallum-5
On Thu, Jun 18, 2015 at 12:24:04AM -0400, Nathaniel McCallum wrote:
> On Thu, 2015-06-18 at 01:13 +0000, Danilo Almeida wrote:
> >                                [...].  Nathaniel, do you have any
> > performance numbers would help the case for the extra effort (and
> > potential risk)?
>
> Well, take common inner-function heap allocations and turn them into
> stack allocations. That is a significant performance gain.

Probably.  A decent heap allocator with per-thread magazines ought to be
good enough in most cases.

> VLAs make perfect sense for operations like fx_cf2 where you need to
> mix two keys together using intermediary buffers. If you validate that
> the input keys are valid types then you know that the buffer for that
> type will be a sensible size to put on the stack. This eliminates two
> unnecessary heap allocations.

Yes, but then, those keys are small.  So VLAs/alloca() here are fine.
alloca() has been around for a long time, no?  You don't need C99 to use
alloca().
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Benjamin Kaduk-2
On Fri, 19 Jun 2015, Nico Williams wrote:

> On Thu, Jun 18, 2015 at 12:24:04AM -0400, Nathaniel McCallum wrote:
> > On Thu, 2015-06-18 at 01:13 +0000, Danilo Almeida wrote:
> > >                                [...].  Nathaniel, do you have any
> > > performance numbers would help the case for the extra effort (and
> > > potential risk)?
> >
> > Well, take common inner-function heap allocations and turn them into
> > stack allocations. That is a significant performance gain.
>
> Probably.  A decent heap allocator with per-thread magazines ought to be
> good enough in most cases.

I'm not sure I follow what point Nico is trying to make, here.  Anyway,
for me, I thought that modern malloc implementations were supposed to be
pretty performant, so that any gains from eliminating calls to malloc
would be lost in the noise relative to the crypto operations involved.

In other words, I'm not convinced that this isn't premature optimization,
as far as claims of performance gains.  (There might still be a reason to
prefer VLAs for code simplicity or readability purposes, of course.)

-Ben
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
Reply | Threaded
Open this post in threaded view
|

Re: C99 Features

Benjamin Kaduk-2
In reply to this post by Nathaniel McCallum-5
Hi Nathaniel,

I think you may be operating under some misconceptions regarding how
kerberos for windows is built.  (More inline)

On Tue, 16 Jun 2015, Nathaniel McCallum wrote:

> On Tue, 2015-06-16 at 01:20 -0400, Greg Hudson wrote:
> > On 06/15/2015 09:06 PM, Nathaniel McCallum wrote:
> > > So how about it? Can MIT start using C99 features?
> >
> > There are four questions here:
> >
> > 1. Can we reasonably require MSVC 2013 for the Windows build?  I
> > don't
> > know of a compelling reason why we can't, but there may be reasons I
> > don't know about.
>
> MSVC 2015 will be released soon. MSVC 2013 seems reasonable given that
> users who don't wish to use it have a free (freedom and beer)
> alternative: clang.

At least some new VS releases require modifications to the KfW build
procedure, in terms of how a shell with the proper build environment is
obtained; they are not drop-in replacements for each other.  The official
KfW builds are currently done with VS 2010, and the windows README should
be sufficient to get someone started with VS 2012.  I think I have been
able to get the build working with VS 2013 but not had time to document
any changes needed; I have not tried VS 2015 at all.

Furthermore, because the MIT Kerberos.exe application uses the Microsoft
Foundation Class library (and possibly other reasons as well), the free
versions of VS are not sufficient to compile KfW; paid versions run
hundreds of dollars.  For that reason, many consumers are still using
older versions, since there is a real barrier to upgrading.  Requiring VS
2013 or newer would require these users to spend money on a new compiler,
which seems like a factor that we should take into account in our
deliberations.

> > 2. Can we abandon MSVC for the Windows build in favor of mingw or
> > clang,
> > in order to get VLA support?  I think the practical answer here is
> > no,
> > in that we don't expect to commit the resources to investigate this
> > possibility in the near future.
>
> I don't think MIT should do this. I think MIT should require MSVC 2013.
> Users who can't use MSVC 2013 should investigate mingw/clang.

KfW is not just normal C application/library code; it integrates with the
aforementioned MFC library, and additionally (as of the 4.1 series) with
more modern UI frameworks provided by Windows.  It is far from clear that
the availability of clang as a compiler would eliminate the need for some
version of VS to be present, for these ancillary libraries and support
tools, and the build infrastructure is not necessarily robust to mixing a
compiler from clang and other tools from elsewhere.

I do not think that "just try clang" is a reasonable thing to ask of
someone else; investigating the compatibility of KfW with clang is going
to require a substantial amount of time.  I don't have that time, myself,
but would be interested in the results if you were to undertake the task.

> > 4. Are there any C99 features we don't want to use, because they
> > don't
> > mesh with our BSD KNF-inspired style or for other reasons?  Going
> > over
> > the specific features you mentioned:
> >
[...]
> > * Does _Bool present any issues if it creeps into public ABIs?  I
> > found
> > http://yarchive.net/comp/linux/bool.html but it's from 2009, and
> > might
> > be specific to the kernel.  It might also just be confusing to have
> > "krb5_boolean" and "bool" in the same code base when they aren't
> > generally the same size (krb5_boolean is a typedef for unsigned int).
>
> I don't care either way on this one.

I would prefer to avoid having _Bool in public ABIs, as there is some
subtlety involved in its use.

> > * Our style guide discourages declaring variables in interior scope.
> > The justification (which predates my involvement) has to do with
> > debugging convenience and limiting function complexity; I personally
> > find that code is a little easier to read if it doesn't have type
> > declarations mixed in with statements.  If there are good reasons to
> > avoid interior scope declarations, those reasons may also apply to
> > mixed
> > declarations and code.
> > http://k5wiki.kerberos.org/wiki/Coding_style/Practices#Local_variable
> > s

Requiring declarations at the function scope can also serve as a motivator
to keep functions short :)

-Ben
_______________________________________________
krbdev mailing list             [hidden email]
https://mailman.mit.edu/mailman/listinfo/krbdev
12