As part of adding support for HTTP/2 stream prioritization, a
DisableClientPriority field will be added to http.Server to allow users
to completely disable client prioritization if desired. When
DisableClientPriority is set to true, HTTP/2 server will revert back to
the old behavior where streams are processed in a round-robin manner.
For golang/go#75500
Change-Id: Ida083b3ac17a953e5ddb3ad7ab8a81f9cde2bfc1
Reviewed-on: https://go-review.googlesource.com/c/net/+/737521
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Nicholas Husin <husin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
To make sure that clients do not unnecessarily send RFC 7540 priority
signals when it would be treated as a no-op, this change makes it so
that our server always sends SETTINGS_NO_RFC7540_PRIORITIES in our
SETTINGS frame when our write scheduler is set to anything other than
the RFC 7540 write scheduler.
For golang/go#75500
Change-Id: I7a54251022087319999deda7efb663f8b251aa95
Reviewed-on: https://go-review.googlesource.com/c/net/+/729141
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Nicholas Husin <husin@google.com>
In RFC 9218, streams are non-incremental by default, meaning that they
are processed one-by-one to completion. This behavior is the opposite of
our current default of handling streams in a round-robin manner.
This might cause a surprising behavior change once we make the RFC 9218
priority scheduler the default write scheduler for most users (we assume
that most users will not be sending RFC 9218 priority signals, at least
initially). To avoid surprising users with such a behavior change, this
CL makes it so that the streams are only made non-incremental once there
has been a clear signal that the end-user is aware of RFC 9218.
For golang/go#75500
Change-Id: Ibd22cb279c43de0190962904c3809007447a5fe3
Reviewed-on: https://go-review.googlesource.com/c/net/+/729140
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Nicholas Husin <husin@google.com>
RFC 9218 allows HTTP/2 stream priority to be set in two ways: via
PRIORITY_UPDATE frame and via header field. This change adds support for
the latter method.
As part of supporting priority adjustment via header field, this CL
also makes sure to look for the existence of an intermediary. If an
intermediary exists, default priority will be used for all streams to
ensure fairness between multiple clients who could be using the same
intermediary.
For golang/go#75500
For golang/go#75936
Change-Id: I6dc409b650fd52fa192d771a16b7a4ac5e51c9aa
Reviewed-on: https://go-review.googlesource.com/c/net/+/729120
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Nicholas Husin <husin@google.com>
This change adds initial support for the PRIORITY_UPDATE frame
introduced in RFC 9218.
Clients can now use a new exported function to write PRIORITY_UPDATE
frames easily. However, sending PRIORITY_UPDATE frames to the server
does not currently cause any behavior changes: we only use
PRIORITY_UPDATE frames to adjust stream priority when the RFC 9218 write
scheduler is being used for a particular connection. However, this
scheduler is not currently usable yet from any configuration surfaces
exposed to the user.
For golang/go#75500
Change-Id: Ie2c821cb0d2faa6e942e209e11638f190fc98e2b
Reviewed-on: https://go-review.googlesource.com/c/net/+/705917
Reviewed-by: Nicholas Husin <husin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
requestBody.Read avoids panicking when requestBody.conn is unset
and we are in tests. This seems like an excellent way to hide a
non-test panic. Drop the exceptional path.
This path was introduced in golang.org/cl/31636 to support a
test verifying that concurrently closing and reading from a
Request.Body does not race. Rewrite this test to use a real
server handler.
Change-Id: I778e78ff9ab45e248769557fff94d17940eb7a18
Reviewed-on: https://go-review.googlesource.com/c/net/+/701000
Auto-Submit: Nicholas Husin <nsh@golang.org>
Reviewed-by: Nicholas Husin <husin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Nicholas Husin <nsh@golang.org>
Replace ad-hoc pre-synctest synchronization with testing/synctest.
Many of the http2 package tests use a custom-built fake time
implementation and a idleness-detection mechanism based on parsing
goroutine stack dumps. Experience with this approach to testing
eventually led to the development of the testing/synctest package.
Switch over to testing/synctest.
The synctest package became available as an experiment in Go 1.24
(only when GOEXPERIMENT=synctest is set), and was fully released
with some API changes in Go 1.25.
- Use the released synctest API on Go 1.25.
- Use the experimental API (synctest.Run) on Go 1.24 when
GOEXPERIMENT=synctest is set. (Note that we set this on trybots.)
- Skip tests on Go 1.24 when GOEXPERIMENT=synctest is not set.
The x/net module requires go1.24, so older versions
can be disregarded.
Change-Id: Ifc13bfdd9bdada6c016730a78bd5972a5193ee30
Reviewed-on: https://go-review.googlesource.com/c/net/+/700996
Reviewed-by: Nicholas Husin <husin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Nicholas Husin <nsh@golang.org>
Allow net/http to pass unencrypted net.Conns to Server/Transport.
We don't have an existing way to pass a conn other than a *tls.Conn
into this package, so (ab)use TLSNextProto to pass unencrypted
connections:
The http2 package adds an "unencrypted_http2" entry to the
TLSNextProto maps. The net/http package calls this function
with a *tls.Conn wrapping a net.Conn with an UnencryptedNetConn
method returning the underlying, unencrypted net.Conn.
For golang/go#67816
Change-Id: I31f9c1ba31a17c82c8ed651382bd94193acf09b9
Reviewed-on: https://go-review.googlesource.com/c/net/+/625175
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Drop a trivial "gate" type used in exactly one location,
replacing it with an equivalent channel operation.
We've been using a "gate" with a different definition in
the quic package; dropping this hapax legomenon type from
the http2 package frees up the name if we want to use the
same gate here.
Change-Id: Id9c7d05daf7ee920c38090df960822fcc1168a4d
Reviewed-on: https://go-review.googlesource.com/c/net/+/584896
Reviewed-by: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
When closing a connection because a stream contained a request we
didn't like (for example, because the request headers exceed
the maximum we will accept), set the LastStreamID in the GOAWAY
frame to include the offending stream. This informs the client
that retrying the request is unlikely to succeed, and avoids
retry loops.
This change requires passing the stream ID of the offending
stream from Framer.ReadFrame up to the caller. The most sensible
way to do this would probably be in the error. However,
ReadFrame currently returns a defined error type for
connection-ending errors (ConnectionError), and that type is a
uint32 with no place to put the stream ID. Rather than changing
the returned errors, ReadFrame now returns an error along with
a non-nil Frame containing the stream ID, when a stream is
responsible for a connection-ending error.
For golang/go#66668
Change-Id: Iba07ccbd70ab4939aa56903605474d01703ac6e4
Reviewed-on: https://go-review.googlesource.com/c/net/+/576756
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
CL 46008 fixedgolang/go#20704 by not recycling the responseWriterState
if any previous Write call failed, as there could be outstanding
goroutines referencing the responseWriterState memory.
More recently, CL 467355 fixed a variant of the same issue by not
referencing that memory after exiting the Write call. This fix
supersedes the fix in CL 46008, as it is more general and does not
require the caller to know whether any previous Write calls failed.
This CL partially reverts CL 46008 just leaving the test case to ensure
that golang/go#20704 does not regress.
Change-Id: I18ea4d27420265a94cc7af21f1dffa3f7dc3bd34
Reviewed-on: https://go-review.googlesource.com/c/net/+/534315
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
Commit-Queue: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
After CL 534215 was merged to fix a CVE it introduced
an underflow when we try to decrement sc.curHandlers
in handlerDone.
The func startPush calls runHandler without incrementing
curHandlers. Seems to only affect users of http.Pusher.
For golang/go#63511
Change-Id: Ic537c27c9945c2c2d4306ddb04e9527b65cee320
GitHub-Last-Rev: 249fe55f75
GitHub-Pull-Request: golang/net#197
Reviewed-on: https://go-review.googlesource.com/c/net/+/535595
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Mauri de Souza Meneguzzo <mauri870@gmail.com>
The priority scheduler allows stream starvation (see golang/go#58804)
and is CPU intensive. In addition, the RFC 7540 prioritization
scheme it implements was deprecated in RFC 9113 and does not appear
to have ever had significant adoption.
Add a simple round-robin scheduler and enable it by default.
For golang/go#58804
Change-Id: I5c5143aa9bc339fc0894f70d773fa7c0d7d87eef
Reviewed-on: https://go-review.googlesource.com/c/net/+/478735
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
A server handler can close an inbound Request.Body to indicate that it
is not interested in the remainder of the request body.
Equivalently, a client can close a Response.Body indicate that it is
not interesed in the remainder of the response body.
In both cases, if we receive DATA frames from the peer for the stream,
we should return connection-level flow control credit for the discarded data.
We do not return stream-level flow control, since we don't want to unblock
further sends of data that we're just going to discard.
Closing either a Response.Body or an inbound Request.Body results in a
pipe.BreakWithError. Reads from a broken pipe fail immediately.
Previously, writes to a broken pipe would succeed, discarding the written
data and incrementing the pipe's unread count. Silently discarding
data written to a broken pipe results in both the Transport and Server
failing to detect the condition where data has been discarded.
Change pipes to return an error when writing to a broken pipe.
Change transportResponseBody.Close to break the response body before
returning flow control credit for unread data in the pipe, avoiding
a race condition where data is added to the pipe in between the
return of flow control credit and the pipe breaking.
Change the Server to treat an error writing to the inbound request
body as an expected condition (since this only happens when a
handler closes the request body), returning connection-level
flow control credit for the discarded data.
Fixesgolang/go#57578
Change-Id: I1ed4ea9865818f9c7d7eb4500edfd7556e3cbcbf
Reviewed-on: https://go-review.googlesource.com/c/net/+/475135
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
When writing data frames, encode the frame on the serve goroutine
rather than in writeFrameAsync to avoid referencing stream data
(originating from a ResponseWriter.Write call) after the Write
has returned.
Fixesgolang/go#58446
Change-Id: I866a7351c90ef122e506b333151f98a455a64953
Reviewed-on: https://go-review.googlesource.com/c/net/+/467355
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Add a new inflow type for tracking inbound flow control.
An inflow tracks both the window sent to the peer, and the
window we are willing to send. Updates are accumulated and
sent in a batch when the unsent window update is large
enough.
This change makes both the client and server use the same
algorithm to decide when to send window updates. This should
slightly reduce the rate of updates sent by the client, and
significantly reduce the rate sent by the server.
Fix a client flow control tracking bug: When processing data
for a canceled stream, the record of flow control consumed
by the peer was not updated to account for the discard
stream.
Fixesgolang/go#28732Fixesgolang/go#56558
Change-Id: Id119d17b84b46f3dc2719f28a86758d9a10085d9
Reviewed-on: https://go-review.googlesource.com/c/net/+/448155
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
The canonical header cache is a per-connection cache mapping header
keys to their canonicalized form. (For example, "foo-bar" => "Foo-Bar").
We limit the number of entries in the cache to prevent an attacker
from consuming unbounded amounts of memory by sending many unique
keys, but a small number of very large keys can still consume an
unreasonable amount of memory.
Track the amount of memory consumed by the cache and limit it based
on memory rather than number of entries.
Thanks to Josselin Costanzi for reporting this issue.
For golang/go#56350
Change-Id: I41db4c9823ed5bf371a9881accddff1268489b16
Reviewed-on: https://go-review.googlesource.com/c/net/+/455635
Reviewed-by: Jenny Rakoczy <jenny@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Add support for handling of SETTINGS_HEADER_TABLESIZE in SETTINGS
frames.
Add http2.Transport.MaxDecoderHeaderTableSize to set the advertised
table size for new client connections. Add
http2.Transport.MaxEncoderHeaderTableSize to cap the accepted size for
new client connections.
Add http2.Server.MaxDecoderHeaderTableSize and MaxEncoderHeaderTableSize
to do the same on the server.
Fixesgolang/go#29356Fixesgolang/go#56054
Change-Id: I16ae0f84b8527dc1e09dfce081e9f408fd514513
Reviewed-on: https://go-review.googlesource.com/c/net/+/435899
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Joedian Reid <joedian@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
After sending a GOAWAY with NO_ERROR, we should discard all frames
for streams with larger identifiers than the last stream identifier
in the GOAWAY frame. We weren't discarding RST_STREAM frames, which
could cause us to incorrectly detect a protocol error when handling
a RST_STREAM for a discarded stream.
Hoist post-GOAWAY frame discarding higher in the loop rather than
handling it on a per-frame-type basis.
We are also supposed to count discarded DATA frames against
connection-level flow control, possibly sending WINDOW_UPDATE
messages to return the flow control. We weren't doing this;
this is now fixed.
Fixesgolang/go#55846
Change-Id: I7603a529c00b8637e648eee9cc4608fb5fa5199b
Reviewed-on: https://go-review.googlesource.com/c/net/+/434909
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: LI ZHEN <mr.imuz@gmail.com>
Reviewed-by: Antonio Ojea <antonio.ojea.garcia@gmail.com>
CL 432038 reduces sending WindowUpdates by introducing a threshold. Once
the remaining bytes are below the threshold, a single WindowUpdate is
sent to reset the amount back to the maximum amount configured.
The window increment size for a stream is calculated from:
sc.srv.initialStreamRecvWindowSize() - st.inflow.available()
Where (*flow).available is defined as:
func (f *flow) available() int32 {
n := f.n
if f.conn != nil && f.conn.n < n {
n = f.conn.n
}
return n
}
When f.conn.c < f.n, it gets a bigger increment size. It should be
calculated from:
sc.srv.initialStreamRecvWindowSize() - st.inflow.n
While we're here, remove an unnecessary type conversion too.
Updates golang/go#56315.
Change-Id: I4b26b27e4c5c5cd66e6a32b152d68f304adc65d8
GitHub-Last-Rev: 02fc09c1e7
GitHub-Pull-Request: golang/net#155
Reviewed-on: https://go-review.googlesource.com/c/net/+/444816
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
This rolls-forward CL 150197 with an added fix for
TestProtocolErrorAfterGoAway.
Rather than send a WindowUpdate on every chunk of bytes read, allow them
to collect until we go past half the configured window size. Once the
threshold is reached, send a single WindowUpdate to reset the amount
back to the maximum amount configured.
Fixesgolang/go#28732
Change-Id: Icee93dedf68d6166aa6fe0c3845d717e66586e73
Reviewed-on: https://go-review.googlesource.com/c/net/+/432038
Run-TryBot: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Tatiana Bradley <tatiana@golang.org>
