[sudo-users] file ulimit not set correctly

[Todd C. Miller]

On Thu, 05 Jul 2012 23:43:40 +0200, Thomas Falkenberg wrote:

> you are right. I have set a too big file ulimit in /etc/security/limits.
> AIX handels it as unlimited. command "ulimit -f" says "unlimited".
> In my opinion it would be a nice feature, if the ulimit is set higher than 
> allowed erroneous, that sudo evaluates it as us unlimited (such as AIX) or 
> as the maximum value 2G - 512blocks (1 TB), but not as zero.

I suspect what really is happening is that AIX simply ignores the
value that is out of range and inherits the ulimit that was set for
root, which in this case is unlimited (-1).

Setting the limit to zero as sudo currently does is a bug, it should
not attempt to set the limit if sudo was unable to retrieve a value
from /etc/security/limits.

The following patch fixes this.

 - todd

diff -r beb8afa0beb2 common/aix.c
--- common/aix.c	Thu Jul 05 09:34:20 2012 -0400
+++ common/aix.c	Fri Jul 06 10:05:06 2012 -0400
@@ -108,9 +108,10 @@
 	    else
 		rlim.rlim_cur = rlim.rlim_max;	/* soft not specd, use hard */
 	} else {
-	    /* No hard limit set, try soft limit. */
-	    if (aix_getlimit(user, aix_limits[n].soft, &val) == 0)
-		rlim.rlim_cur = val == -1 ? RLIM64_INFINITY : val * aix_limits[n].factor;
+	    /* No hard limit set, try soft limit, if it exists. */
+	    if (aix_getlimit(user, aix_limits[n].soft, &val) == -1)
+		continue;
+	    rlim.rlim_cur = val == -1 ? RLIM64_INFINITY : val * aix_limits[n].factor;
 
 	    /* Set hard limit per AIX /etc/security/limits documentation. */
 	    switch (aix_limits[n].resource) {