stream: remove {writeableState/readableState}.length #12857

calvinmetcalf · 2017-05-05T12:52:43Z

As part of the readableState/writableState mega issue #445, this
removes all of the references to .length on those properties and
replaces them with a readLength and writeLength getter.

cc @nodejs/streams

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

streams, tls, net

mscdex · 2017-05-05T15:49:47Z

As with #12860, is performance impacted by these changes?

Trott · 2017-05-05T21:55:22Z

Should documentation be added for the new property?

jasnell

The actual code change looks fine to me. I agree with @mscdex that a benchmark run would be good so we can see the perf impact, and @Trott is right that this needs docs added.

calvinmetcalf · 2017-05-08T12:03:58Z

do we want to document these properties? would it make sense to underscore prefix them and not publicly document them?

mcollina · 2017-05-11T14:04:04Z

I will run the benchmarks next week when I am back.

calvinmetcalf · 2017-05-11T14:44:24Z

@mcollina great thanks! we don't have a rush on this one, it's been open for a while so we should fix it right.

mcollina · 2017-05-17T09:42:50Z

@calvinmetcalf can you rebase? It's failing if I try to apply this.

calvinmetcalf · 2017-05-17T11:59:44Z

ok I'll rebase

jasnell · 2017-06-13T20:21:11Z

lib/_stream_duplex.js

@@ -61,6 +61,13 @@ function Duplex(options) {
  this.once('end', onend);
 }

+
+Object.defineProperty(Duplex.prototype, 'writeLength', {
+  get: function() {


nit: these could just be get() { ... }

mscdex · 2017-06-13T21:26:08Z

Any benchmark results for this yet?

mcollina · 2017-06-15T13:23:19Z

@mscdex my fault. I've been busy and I did not have time. I'll get things done next week.

mcollina · 2017-06-20T07:19:30Z

I have pulled this on top of current master, and here are the result:

$ cat compare-pr-12857.csv | Rscript benchmark/compare.R
                                                   improvement confidence    p.value
 net/net-c2s-cork.js dur=5 type="buf" len=1024          0.05 %            0.92585453
 net/net-c2s-cork.js dur=5 type="buf" len=128           0.01 %            0.98394598
 net/net-c2s-cork.js dur=5 type="buf" len=16            0.11 %            0.64262178
 net/net-c2s-cork.js dur=5 type="buf" len=32           -0.63 %            0.49813748
 net/net-c2s-cork.js dur=5 type="buf" len=4             0.69 %            0.22906509
 net/net-c2s-cork.js dur=5 type="buf" len=512          -0.68 %            0.11614061
 net/net-c2s-cork.js dur=5 type="buf" len=64           -0.10 %            0.88165041
 net/net-c2s-cork.js dur=5 type="buf" len=8             0.22 %            0.49318844
 net/net-c2s.js dur=5 type="asc" len=102400             0.12 %            0.79652780
 net/net-c2s.js dur=5 type="asc" len=16777216           0.17 %            0.69953088
 net/net-c2s.js dur=5 type="buf" len=102400            -0.03 %            0.95501291
 net/net-c2s.js dur=5 type="buf" len=16777216          -0.18 %            0.60032466
 net/net-c2s.js dur=5 type="utf" len=102400            -0.71 %            0.30724140
 net/net-c2s.js dur=5 type="utf" len=16777216           0.13 %            0.71738184
 net/net-pipe.js dur=5 type="asc" len=102400           -0.38 %            0.47130806
 net/net-pipe.js dur=5 type="asc" len=16777216         -0.50 %            0.22687420
 net/net-pipe.js dur=5 type="buf" len=102400            0.15 %            0.73656293
 net/net-pipe.js dur=5 type="buf" len=16777216          0.34 %            0.34467836
 net/net-pipe.js dur=5 type="utf" len=102400           -0.13 %            0.76703831
 net/net-pipe.js dur=5 type="utf" len=16777216         -0.06 %            0.82737525
 net/net-s2c.js dur=5 type="asc" len=102400             0.37 %            0.34771981
 net/net-s2c.js dur=5 type="asc" len=16777216           1.08 %          * 0.01600552
 net/net-s2c.js dur=5 type="buf" len=102400            -0.10 %            0.76992876
 net/net-s2c.js dur=5 type="buf" len=16777216          -0.01 %            0.97334494
 net/net-s2c.js dur=5 type="utf" len=102400             0.44 %            0.53842239
 net/net-s2c.js dur=5 type="utf" len=16777216           0.16 %            0.63315209
 net/tcp-raw-c2s.js dur=5 type="asc" len=102400        -0.20 %            0.67051959
 net/tcp-raw-c2s.js dur=5 type="asc" len=16777216       0.21 %            0.30793474
 net/tcp-raw-c2s.js dur=5 type="buf" len=102400        -0.29 %            0.28911115
 net/tcp-raw-c2s.js dur=5 type="buf" len=16777216      -0.04 %            0.86124577
 net/tcp-raw-c2s.js dur=5 type="utf" len=102400        -0.30 %            0.57333080
 net/tcp-raw-c2s.js dur=5 type="utf" len=16777216      -0.18 %            0.60026095
 net/tcp-raw-pipe.js dur=5 type="asc" len=102400        1.06 %            0.77474299
 net/tcp-raw-pipe.js dur=5 type="asc" len=16777216      3.02 %            0.34301122
 net/tcp-raw-pipe.js dur=5 type="buf" len=102400       -6.20 %            0.11493755
 net/tcp-raw-pipe.js dur=5 type="buf" len=16777216      0.53 %            0.86680583
 net/tcp-raw-pipe.js dur=5 type="utf" len=102400        4.27 %            0.26626613
 net/tcp-raw-pipe.js dur=5 type="utf" len=16777216     -0.77 %            0.80117040
 net/tcp-raw-s2c.js dur=5 type="asc" len=102400         0.70 %            0.10070950
 net/tcp-raw-s2c.js dur=5 type="asc" len=16777216       0.36 %            0.06191144
 net/tcp-raw-s2c.js dur=5 type="buf" len=102400         0.05 %            0.91598591
 net/tcp-raw-s2c.js dur=5 type="buf" len=16777216       0.05 %            0.82678721
 net/tcp-raw-s2c.js dur=5 type="utf" len=102400         0.60 %            0.32760427
 net/tcp-raw-s2c.js dur=5 type="utf" len=16777216      -0.21 %            0.28876196

IMHO we can land this, there are no regressions.

calvinmetcalf · 2017-06-20T12:12:15Z

ok let me fix the nits and rebase

calvinmetcalf · 2017-06-20T13:02:54Z

ok done

mscdex · 2017-06-20T15:12:24Z

Can we use 'readable' and 'writable' as the prefixes instead of just 'read' and 'write' respectively, to further reduce the likelihood of a clash with userland?

I'm still -1 on adding all of these individual properties to everyone's streams. I would rather see something like an object housing all of these...

mcollina · 2017-06-20T15:32:42Z

Currently we are creating an accessory object (the state) for every Readable and Writable, and two for Duplex (and descendants). This is already a bottleneck, so I'm -1 on adding a new object for both Readable and Writable.

BTW, I'm fine with documenting the _ properties, and/or remapping those.

jasnell · 2017-06-20T15:54:51Z

If we're going to document the existing ones, I'd prefer to get rid of the _ prefix.

mcollina · 2017-06-20T15:59:45Z

@jasnell that is not feasible either, as everyone is using those _ properties.

mcollina · 2017-08-04T07:17:53Z

@mscdex are you still -1 on this?

This might require some more reviews/approvals.. @nodejs/ctc as we discussed in the last meeting.

mscdex · 2017-08-04T07:37:24Z

@mcollina Yes, I am still -1 in general to these and the related changes from other PRs.

addaleax · 2017-11-28T01:31:34Z

@calvinmetcalf Do you think you could rebase this? Otherwise I guess one of us could do that.

mcollina · 2017-11-28T02:39:16Z

I can do this in a week, or if someone is willing to help please go ahead and land. Il giorno mar 28 nov 2017 alle 08:02 Anna Henningsen < notifications@github.com> ha scritto:

…

@calvinmetcalf <https://github.com/calvinmetcalf> Do you think you could rebase this? Otherwise I guess one of us could do that. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12857 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADL46E03AzAdukGrmEWyFKEqE79BJ5Bks5s62KrgaJpZM4NR4Cu> .

As part of the readableState/writableState mega issue nodejs#445, this removes all of the references to .length on those properties and replaces them with a readableLength and writableLength getter.

mcollina · 2017-12-18T11:11:03Z

CI before landing:

https://ci.nodejs.org/job/node-test-pull-request/12167/

mcollina · 2017-12-18T13:43:01Z

Landed as 36ffa21.

As part of the readableState/writableState mega issue #445, this removes all of the references to .length on those properties and replaces them with a readableLength and writableLength getter. See: #445 PR-URL: #12857 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

Notable change: * async_hooks: - deprecate AsyncHooks Sensitive API and runInAsyncIdScope. Neither API were documented. (Andreas Madsen) #16972 * deps: - update nghttp2 to 1.29.0 (James M Snell) #17908 - upgrade npm to 5.6.0 (Kat Marchán) #17535 - cherry-pick 50f7455 from upstream V8 (Michaël Zasso) #16591 * events: - remove reaches into _events internals (Anatoli Papirovski) #17440 * http: - add rawPacket in err of `clientError` event (XadillaX) #17672 * http2: - implement maxSessionMemory (James M Snell) #17967 - add initial support for originSet (James M Snell) #17935 - add altsvc support (James M Snell) #17917 - perf_hooks integration (James M Snell) #17906 * net: - remove Socket.prototype.write (Anna Henningsen) #17644 - remove Socket.prototype.listen (Ruben Bridgewater) #13735 * repl: - show lexically scoped vars in tab completion (Michaël Zasso) #16591 * stream: - rm {writeable/readable}State.length (Calvin Metcalf) #12857 - add flow and buffer properties to streams (Calvin Metcalf) #12855 * util: - allow wildcards in NODE_DEBUG variable (Tyler) #17609 * zlib: - add ArrayBuffer support (Jem Bezooyen) #16042 * Addedew collaborator** - [starkwang](https://github.com/starkwang) Weijia Wang * Addedew TSC member** - [danbev](https://github.com/danbev) Daniel Bevenius PR-URL: #18069

Notable change: * async_hooks: - deprecate AsyncHooks Sensitive API and runInAsyncIdScope. Neither API were documented. (Andreas Madsen) #16972 * deps: - update nghttp2 to 1.29.0 (James M Snell) #17908 - upgrade npm to 5.6.0 (Kat Marchán) #17535 - cherry-pick 50f7455 from upstream V8 (Michaël Zasso) #16591 * events: - remove reaches into _events internals (Anatoli Papirovski) #17440 * http: - add rawPacket in err of `clientError` event (XadillaX) #17672 * http2: - implement maxSessionMemory (James M Snell) #17967 - add initial support for originSet (James M Snell) #17935 - add altsvc support (James M Snell) #17917 - perf_hooks integration (James M Snell) #17906 - Refactoring and cleanup of Http2Session and Http2Stream destroy (James M Snell) #17406 * net: - remove Socket.prototype.write (Anna Henningsen) #17644 - remove Socket.prototype.listen (Ruben Bridgewater) #13735 * repl: - show lexically scoped vars in tab completion (Michaël Zasso) #16591 * stream: - rm {writeable/readable}State.length (Calvin Metcalf) #12857 - add flow and buffer properties to streams (Calvin Metcalf) #12855 * util: - allow wildcards in NODE_DEBUG variable (Tyler) #17609 * zlib: - add ArrayBuffer support (Jem Bezooyen) #16042 * Addedew collaborator** - [starkwang](https://github.com/starkwang) Weijia Wang * Addedew TSC member** - [danbev](https://github.com/danbev) Daniel Bevenius PR-URL: #18069

As part of the readableState/writableState mega issue #445, this removes all of the references to .length on those properties and replaces them with a readableLength and writableLength getter. See: nodejs/node#445 PR-URL: nodejs/node#12857 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Matteo Collina <matteo.collina@gmail.com>

calvinmetcalf added net stream tls labels May 5, 2017

nodejs-github-bot added the lib / src label May 5, 2017

Fishrock123 added this to Work In Progress / Has PR in Streams: _readableState and _writableState May 5, 2017

jasnell requested changes May 7, 2017

View changes

calvinmetcalf force-pushed the read-write-length branch from c1b27b7 to 23f2ca9 Compare May 17, 2017

jasnell reviewed Jun 13, 2017

View changes

jasnell approved these changes Jun 13, 2017

View changes

calvinmetcalf force-pushed the read-write-length branch from 23f2ca9 to 606964b Compare Jun 20, 2017

mcollina added this to the 9.0.0 milestone Jul 20, 2017

addaleax removed their assignment Nov 28, 2017

MylesBorins force-pushed the master branch from b7405ab to 7f086dd Compare Dec 8, 2017

mcollina force-pushed the read-write-length branch from ae9f3cd to 7753a77 Compare Dec 11, 2017

stream: rm {writeable/readable}State.length

915dab7

As part of the readableState/writableState mega issue nodejs#445, this removes all of the references to .length on those properties and replaces them with a readableLength and writableLength getter.

mcollina force-pushed the read-write-length branch from 7753a77 to 915dab7 Compare Dec 18, 2017

mcollina closed this Dec 18, 2017

mcollina deleted the read-write-length branch Dec 18, 2017

MylesBorins mentioned this pull request Jan 10, 2018

v9.4.0 proposal #18069

Merged

MylesBorins added the dont-land-on-v8.x label Aug 2, 2018

Trott moved this from Work In Progress / Has PR to Done in Streams: _readableState and _writableState May 1, 2019

nodejs / node Public

stream: remove {writeableState/readableState}.length #12857

stream: remove {writeableState/readableState}.length #12857

calvinmetcalf commented May 5, 2017 •

edited

mscdex commented May 5, 2017

Trott commented May 5, 2017

jasnell left a comment

calvinmetcalf commented May 8, 2017

mcollina commented May 11, 2017

calvinmetcalf commented May 11, 2017

mcollina commented May 17, 2017

calvinmetcalf commented May 17, 2017

jasnell Jun 13, 2017

mscdex commented Jun 13, 2017

mcollina commented Jun 15, 2017

mcollina commented Jun 20, 2017

calvinmetcalf commented Jun 20, 2017

calvinmetcalf commented Jun 20, 2017

mscdex commented Jun 20, 2017

mcollina commented Jun 20, 2017

jasnell commented Jun 20, 2017

mcollina commented Jun 20, 2017

mcollina commented Aug 4, 2017

mscdex commented Aug 4, 2017

addaleax commented Nov 28, 2017

mcollina commented Nov 28, 2017

mcollina commented Dec 18, 2017

mcollina commented Dec 18, 2017

nodejs / node Public

stream: remove {writeableState/readableState}.length #12857

stream: remove {writeableState/readableState}.length #12857

Conversation

calvinmetcalf commented May 5, 2017 • edited

Checklist

Affected core subsystem(s)

mscdex commented May 5, 2017

Trott commented May 5, 2017

jasnell left a comment

calvinmetcalf commented May 8, 2017

mcollina commented May 11, 2017

calvinmetcalf commented May 11, 2017

mcollina commented May 17, 2017

calvinmetcalf commented May 17, 2017

jasnell Jun 13, 2017

Choose a reason for hiding this comment

mscdex commented Jun 13, 2017

mcollina commented Jun 15, 2017

mcollina commented Jun 20, 2017

calvinmetcalf commented Jun 20, 2017

calvinmetcalf commented Jun 20, 2017

mscdex commented Jun 20, 2017

mcollina commented Jun 20, 2017

jasnell commented Jun 20, 2017

mcollina commented Jun 20, 2017

mcollina commented Aug 4, 2017

mscdex commented Aug 4, 2017

addaleax commented Nov 28, 2017

mcollina commented Nov 28, 2017

mcollina commented Dec 18, 2017

mcollina commented Dec 18, 2017

calvinmetcalf commented May 5, 2017 •

edited