cmd: itest: correct calculus for long format

itest shell command doesn't work correctly in long format when doing comparaison due to wrong mask value calculus that overflow on 32 bits values. Signed-off-by: Sebastien Colleur <sebastienx.colleur@intel.com> Reviewed-by: Simon Glass <sjg@chromium.org>

cmd: itest: correct calculus for long format
itest shell command doesn't work correctly in long format when doing comparaison due to wrong mask value calculus that overflow on 32 bits values. Signed-off-by: Sebastien Colleur <sebastienx.colleur@intel.com> Reviewed-by: Simon Glass <sjg@chromium.org>
Sebastien Colleur · Tom Rini
1 parent 2cfe312258
Showing 1 changed file with 2 additions and 1 deletions Side-by-side Diff
cmd/itest.c
@@ -80,7 +80,8 @@
 		l = simple_strtoul(s, NULL, 16);
 	}
  
-	return l & ((1UL << (w * 8)) - 1);
+	/* avoid overflow on mask calculus */
+	return (w >= sizeof(long)) ? l : (l & ((1UL << (w * 8)) - 1));
 }
  
 static char * evalstr(char *s)
...	...	@@ -80,7 +80,8 @@
80	80	l = simple_strtoul(s, NULL, 16);
81	81	}
82	82
83		- return l & ((1UL << (w * 8)) - 1);
	83	+ /* avoid overflow on mask calculus */
	84	+ return (w >= sizeof(long)) ? l : (l & ((1UL << (w * 8)) - 1));
84	85	}
85	86
86	87	static char * evalstr(char *s)