ES5conform Issue Tracker Rss Feed

Created Issue: 15.12.3-11-6.js - typo precodition [30361]

efreeti — Thu, 17 Mar 2011 09:35:57 GMT

I'm emulating the run of tests in my unit test system and do not take into account situation when the precondition is not defined. This test - TestCases/chapter15/15.12/15.12.3/15.12.3-11-6.js - fails and after looking at the code I see that there is a precondition function just there is the typo in the property name - 'precodition'.

Commented Issue: Test 7.8.4-1-s does not correctly check whether octal escape sequences are allowed [28578]

mmaly — Fri, 04 Feb 2011 22:36:48 GMT

The spec does not allow octal escape sequences within strict mode code. Test 7.8.4-1-s is designed to check this - however, I believe it fails to do so.

The code in question is this: eval(' "asterisk: \052"; "use strict";');

The eval function takes a single string and by the time it reaches that point, the escape sequences have been eliminated. Thus the octal escape sequence appears *outside* strict mode code (i.e. in the context of the testcase function) and thus it should be allowed.

Also debatable is whether octal escape sequences should be disallowed before the "use strict" directive is encountered. This seems like it would require the lexer to do a second pass over the code solely to resolve escape sequences.
Comments: ** Comment from web user: mmaly **

We should commit the relevant part of the patch attached to http://es5conform.codeplex.com/workitem/29141.

Commented Issue: Deleting a variable in strict mode produces a SyntaxError, not a ReferenceError (tests 11.4.1-5-1-s, 11.4.1-5-2-s, 11.4.1-5-3-s) [29084]

jwalden — Mon, 03 Jan 2011 22:29:24 GMT

The tests 11.4.1-5-1-s, 11.4.1-5-2-s and 11.4.1-5-3-s all assume that deleting a variable in strict mode produces a ReferenceError exception. Section 11.4.1 of the specification contradicts this as it specifies a SyntaxError should be thrown instead.
Comments: ** Comment from web user: jwalden **

I sent that es5-discuss mail:

https://mail.mozilla.org/pipermail/es5-discuss/2011-January/003862.html

Commented Issue: Deleting a variable in strict mode produces a SyntaxError, not a ReferenceError (tests 11.4.1-5-1-s, 11.4.1-5-2-s, 11.4.1-5-3-s) [29084]

jwalden — Mon, 03 Jan 2011 21:12:48 GMT

The colorable case involves a host object with a [[HasProperty]] hook that has side effects or may throw an exception other than a SyntaxError, then supplied as the object for a with clause, thus making the SyntaxError for deleting a name potentially not reachable if the delete expression is reached at runtime. ES6's proposed proxy specification would enable creation of such an object. The spec says, "An early error is an error that can be detected and reported prior to the evaluation of any construct in the Program containing the error." "can be detected and reported prior to..." is arguably under-defined enough to permit the host object snafu. As I said, I don't like this case, and I think it worth killing. But there is a case that deleting a name is not an early error.

Commented Issue: Tests 10.6-13-b-3-s and 10.6-13-c-3-s reference nonexistent property "put" [29141]

jwalden — Mon, 03 Jan 2011 21:10:36 GMT

The tests 10.6-13-b-3-s and 10.6-13-c-3-s call Object.getOwnPropertyDescriptor to determine whether "arguments.callee" and "arguments.caller" are configurable. However, they check for the existence of the property "put". The correct property to check for is "set".
Comments: ** Comment from web user: jwalden **

Here are SpiderMonkey's results, for what it's worth, with all my patches (all attached to bugs, most waiting to be reviewed, one almost at the ready-to-attach stage, one very hackish patch that probably needs more work):

We fail these valid tests (tho I believe there's a patch, awaiting my review, to fix all these):

FAIL: 15.4.4.14-1-1 Array.prototype.indexOf applied to undefined throws a TypeError
FAIL: 15.4.4.14-1-2 Array.prototype.indexOf applied to null throws a TypeError
FAIL: 15.5.4.20-1-2 String.prototype.trim throws TypeError when string is null
FAIL: 15.5.4.20-1-1 String.prototype.trim throws TypeError when string is undefined

These tests seem invalid to me if you don't extend ES5:

FAIL: 15.2.3.3-4-188 Object.getOwnPropertyDescriptor returns undefined for non-existent properties on built-ins (Function (instance).name)
FAIL: 15.2.3.3-4-184 Object.getOwnPropertyDescriptor returns undefined for non-existent properties on built-ins (Function.caller)
FAIL: 15.2.3.3-4-183 Object.getOwnPropertyDescriptor returns undefined for non-existent properties on built-ins (Function.arguments)

These tests are nearly-valid, but I just didn't fix them in the patch:

FAIL: 11.4.1-4.a-4-s delete operator throws TypeError when when deleting a non-configurable data property in strict mode (Global.NaN)
FAIL: 11.13.1-4-2-s simple assignment throws TypeError if LeftHandSide is a readonly property in strict mode (Global.NaN)
FAIL: 11.13.1-4-3-s simple assignment throws TypeError if LeftHandSide is a readonly property in strict mode (Global.Infinity)
FAIL: 11.13.1-4-4-s simple assignment throws TypeError if LeftHandSide is a readonly property in strict mode (Global.length)
FAIL: 11.13.1-1-7-s simple assignment throws TypeError if LeftHandSide is a property reference with a primitive base value (this is undefined)
FAIL: 11.13.1-4-27-s simple assignment throws TypeError if LeftHandSide is a readonly property in strict mode (Global.undefined)
PRECONDITION FAILED: 15.4.4.14-9.a-1 Need to define a quality of implementation test that makes sure that it doesn't take forever to process very large but very sparse arrays

These tests are the delete-name-is-early-error disagreement:

TestCases/chapter11/11.4/11.4.1/11.4.1-5-2-s.js Missing test: either misidentifed or could not load because of unexpected syntax error
TestCases/chapter11/11.4/11.4.1/11.4.1-5-3-s.js Missing test: either misidentifed or could not load because of unexpected syntax error
TestCases/chapter11/11.4/11.4.1/11.4.1-5-1-s.js Missing test: either misidentifed or could not load because of unexpected syntax error

This test probably works in a browser, but I generally don't test browser until I have a working patch for a much smaller, less hairy shell environment (plus browser takes a lot longer to build):

FAILED WITH EXCEPTION: WINDOW IS NOT DEFINED: 11.13.1-4-1 simple assignment creates property on the global object if LeftHandSide is an unresolvable reference

So aside from the two tests that expect no Function.caller/Function.arguments (this is outside the spec, which permits additional properties here and doesn't specify that these exist, as Function is not a function created through 13.something), we are in agreement as to what tests are bad and what tests are good. Or, rather, our implementations plus a little manual testing are in agreement.

Commented Issue: Tests 10.6-13-b-3-s and 10.6-13-c-3-s reference nonexistent property "put" [29141]

paulbartrum — Sun, 02 Jan 2011 13:06:11 GMT

Thanks Jeff. With your patch applied my ES5 implementation (http://jurassic.codeplex.com) fails 13 tests (down from 45). Two are known bugs with my implementation - the rest are currently outstanding issues on this issue tracker (6 issues covering 11 tests). By the way, your patch exposed a bug in my strict mode implementation, so thanks for your hard work!

In case you are interested, here are the eleven tests that I believe are still buggy (of course I am only picking up false negatives, not false positives):
11.4.1-4.a-4-s // assumes this refers to global object - http://es5conform.codeplex.com/workitem/29151
11.4.1-5-1-s // assumes delete var produces ReferenceError - http://es5conform.codeplex.com/workitem/29084
11.4.1-5-2-s // assumes delete var produces ReferenceError - http://es5conform.codeplex.com/workitem/29084
11.4.1-5-3-s // assumes delete var produces ReferenceError - http://es5conform.codeplex.com/workitem/29084
11.13.1-1-7-s // assumes this is undefined - http://es5conform.codeplex.com/workitem/29152
11.13.1-4-2-s // gets global object incorrectly - http://es5conform.codeplex.com/workitem/29087
11.13.1-4-27-s // gets global object incorrectly - http://es5conform.codeplex.com/workitem/29087
11.13.1-4-3-s // gets global object incorrectly - http://es5conform.codeplex.com/workitem/29087
11.13.1-4-4-s // gets global object incorrectly - http://es5conform.codeplex.com/workitem/29087
15.2.3.3-4-188 // assumes Function.prototype.name does not exist - http://es5conform.codeplex.com/workitem/28594
15.4.4.14-9.a-1 // placeholder test - http://es5conform.codeplex.com/workitem/29102

Commented Issue: Deleting a variable in strict mode produces a SyntaxError, not a ReferenceError (tests 11.4.1-5-1-s, 11.4.1-5-2-s, 11.4.1-5-3-s) [29084]

paulbartrum — Sun, 02 Jan 2011 06:20:56 GMT

For the record, my reasoning for believing it is an early error is as follows:
* Section 11.4.1 step 5 indicates that deleting a variable in strict mode is a SyntaxError.
* Section 16 says an implementation must treat any syntax error as an early error.
Seems pretty straightforward to me.

Commented Issue: Test 7.8.4-1-s does not correctly check whether octal escape sequences are allowed [28578]

jwalden — Sat, 01 Jan 2011 23:01:30 GMT

http://es5conform.codeplex.com/workitem/29141 has a patch that also fixes this.

I proposed a fix for your suggestion (directives must not contain escapes, ergo "\052"; "use strict"; would not enter strict mode and require reparsing, or carrying seen-octal state forward when parsing the directive prologue -- SpiderMonkey does the latter because it eagerly throws away source) to the es5-discuss list, but discussion fizzled, and eventually I became fed up on waiting for a definitive yes/no as to making that change and just implemented what the spec requires in SpiderMonkey. https://bugzilla.mozilla.org/show_bug.cgi?id=601262 and https://mail.mozilla.org/pipermail/es5-discuss/2010-October/003745.html are the relevant bug and mailing list discussion. I'd still love it if the spec eliminated the need for this pedantry and reparsing!

Commented Issue: The test 15.2.3.4-4-1 assumes a fixed list of properties on the global object [28595]

jwalden — Sat, 01 Jan 2011 22:58:34 GMT

The test 15.2.3.4-4-1 checks the property names of the global object against the following list:
["eval", "parseInt", "parseFloat", "isNaN", "isFinite", "decodeURI", "decodeURIComponent", "encodeURIComponent", "escape", "unescape", "NaN", "Infinity", "undefined"]
This test has two problems:
1. The list above is missing quite a few standard properties (e.g. "encodeURI", "Array", "JSON", "Math", etc)
2. Implementations are allowed to extend the global object and many do (including all the major browsers).
Comments: ** Comment from web user: jwalden **

I fixed the subsetting issue awhile ago, but the "incomplete properties" complaint is still valid.

Commented Issue: Array iteration methods may continue after a reference to the array has been deleted [25430]

jwalden — Sat, 01 Jan 2011 22:56:30 GMT

I think these tests are making an incorrect assertion:

15.4.4.16-7-7
15.4.4.17-7-7
15.4.4.18-7-6
15.4.4.19-8-7
15.4.4.20-9-7
15.4.4.21-9-7
15.4.4.22-9-7

In these tests the object property is being deleted, but that doesn't mean that the value itself is deleted - the method being called (e.g. every or filter) still has a reference to the array it's operating on.
Comments: ** Comment from web user: jwalden **

http://es5conform.codeplex.com/workitem/29141 has a patch that fixes this.

Commented Issue: All "stops calling callbackfn once the array is deleted during the call" tests are wrong [26872]

jwalden — Sat, 01 Jan 2011 22:54:12 GMT

These tests (15.4.4.16-7-7, 15.4.4.17-7-7, 15.4.4.18-7-6, 15.4.4.19-8-7, 15.4.4.20-9-7, 15.4.4.21-9-7, 15.4.4.22-9-7) follow the same pattern (this is reduce) :
function testcase() {
function callbackfn(prevVal, curVal, idx, obj) {
delete o.arr;
return prevVal + curVal;
}

var o = new Object;
o.arr = ["1", 2, 3, 4, 5];
if (o.arr.reduce(callbackfn) === "12") {
return true;
}
}

I think that, in this case, "delete" is misunderstood. The delete operator (ES5 11.4.1) calls the [[delete]] internal method (ES5 8.12.7) which removes from an object a property (if configurable). If this property happens to be an object, there is no reason that this object itself is deleted as long as a reference to this object is kept somewhere. In our case, the "obj" argument of callbackfn and the "this" keyword of reduce are both a reference to the array being traversed.

For that reason, I think that this these tests are wrong and should be removed from the test suite.
Comments: ** Comment from web user: jwalden **

This seems a dup of 25430, fwiw.

Commented Issue: All "stops calling callbackfn once the array is deleted during the call" tests are wrong [26872]

jwalden — Sat, 01 Jan 2011 22:52:58 GMT

http://es5conform.codeplex.com/workitem/29141 has a patch that fixes this, but by adjusting the test to pass, not by deleting it. (I have no particular objection to deleting the test, but since it's basically written, better to coopt it for a useful purpose if possible, I think, even if it's a completely different purpose from the original one.)

Commented Issue: The preconditions for tests 15.12.2-0-3 and 15.12.3-0-3 do not return true [28581]

jwalden — Sat, 01 Jan 2011 22:50:07 GMT

The preconditions for tests 15.12.2-0-3 and 15.12.3-0-3 both return a function and not a boolean.

For example, in test 15.12.2-0-3 the precondition is: "return JSON && Object.getOwnPropertyDescriptor". getOwnPropertyDescriptor is a function and therefore the whole expression returns a function. Since the test driver expects "true" the precondition fails. I believe that in this case the problem is that "fnExists" is missing and the precondition should be "return JSON && fnExists(Object.getOwnPropertyDescriptor)".
Comments: ** Comment from web user: jwalden **

http://es5conform.codeplex.com/workitem/29141 has a patch that also fixes this, although the fix is slightly different from what's suggested here.

Commented Issue: Deleting a variable in strict mode produces a SyntaxError, not a ReferenceError (tests 11.4.1-5-1-s, 11.4.1-5-2-s, 11.4.1-5-3-s) [29084]

jwalden — Sat, 01 Jan 2011 22:44:39 GMT

http://es5conform.codeplex.com/workitem/29141 has a patch that fixes many test suite issues but doesn't address this, because there's a somewhat colorable case that this is not necessarily an early error, and I haven't taken it to es5-discuss yet. But it's worth noting here that SpiderMonkey and (soon, I believe) Nitro will implement this as an early error, even if perhaps that shouldn't play into the calculation of whether these tests should or shouldn't change.

Commented Issue: Tests 15.4.4.21-9-c-ii-4-s and 15.4.4.22-9-c-ii-4-s assert that null should be passed as the this value (should be undefined) [29085]

jwalden — Sat, 01 Jan 2011 22:41:00 GMT

The tests 15.4.4.21-9-c-ii-4-s and 15.4.4.22-9-c-ii-4-s assert that null should be passed as the this value to the callback function for reduce and reduceRight. The spec is inconsistent (it says undefined for reduce and null for reduceRight) but in the latest errata it clarifies that both functions should pass undefined as the this value.
Comments: ** Comment from web user: jwalden **

http://es5conform.codeplex.com/workitem/29141 has a patch that also fixes this, although I believe it's not fixed quite as suggested here.

Commented Issue: The precondition for test 15.4.4.19-9-3 references an undefined variable [29088]

jwalden — Sat, 01 Jan 2011 22:39:49 GMT

The precondition for test 15.4.4.19-9-3 references "a":
return fnExists(Array.prototype.map, Array.isArray(a));
The variable "a" is not yet defined. This causes the precondition to fail.
Comments: ** Comment from web user: jwalden **

http://es5conform.codeplex.com/workitem/29141 has a patch that also fixes this.

Commented Issue: Tests 12.2.1-1-s to 12.2.1-10-s expect an EvalError when it should be a SyntaxError [29089]

jwalden — Sat, 01 Jan 2011 22:39:39 GMT

Tests 12.2.1-1-s to 12.2.1-10-s all expect that declaring a variable called "eval" will throw an EvalError. This is not correct - the spec states that SyntaxError should be thrown instead.
Comments: ** Comment from web user: jwalden **

http://es5conform.codeplex.com/workitem/29141 has a patch that also fixes this.

Commented Issue: Tests 13.1-3-3-s and others are missing a return statement [29100]

jwalden — Sat, 01 Jan 2011 22:39:36 GMT

If the test succeeds, there is no return statement so the function returns undefined. This causes the test to fail.
The following tests are affected:
13.1-3-3-s
13.1-3-4-s
13.1-3-5-s
13.1-3-6-s
13.1-3-9-s
13.1-3-10-s
13.1-3-11-s
13.1-3-12-s

The code:
e instanceof SyntaxError
should be:
return e instanceof SyntaxError
Comments: ** Comment from web user: jwalden **

http://es5conform.codeplex.com/workitem/29141 has a patch that also fixes this, although I'm not sure offhand if it fixes *all* the tests here -- probably, but not assuredly.

Commented Issue: Test 15.3.2.1-11-6-s does not return a value [29103]

jwalden — Sat, 01 Jan 2011 22:38:32 GMT

As far as I can tell, the test must return a value convertible to true in order to succeed. There is no return statement in the test 15.3.2.1-11-6-s, so it returns undefined, which is convertible to false. Therefore the test always fails.
Comments: ** Comment from web user: jwalden **

http://es5conform.codeplex.com/workitem/29141 has a patch that also fixes this.

Commented Issue: Test 15.4.4.17-4-9 incorrectly asserts that Array.prototype.some returns -1 [29143]

jwalden — Sat, 01 Jan 2011 22:37:46 GMT

The test 15.4.4.17-4-9 asserts that Array.prototype.some returns -1 if the length property is zero. This is incorrect; it should return false instead.
Comments: ** Comment from web user: jwalden **

http://es5conform.codeplex.com/workitem/29141 has a patch that also fixes this.