This change adds the flight recorder to the trace package.
Flight recording is a technique in which trace data is kept
in a circular buffer and can be flushed upon request. The
implementation will be added in follow-up CLs.
The flight recorder has already been implemented inside of the
golang.org/x/exp/trace package. This copies the current implementation
and modifies it to work within the runtime/trace package.
The changes include:
This adds the ability for multiple consumers (both the execution
tracer and the flight recorder) to subscribe to tracing events. This
change allows us to add multiple consumers without making major
modifications to the runtime. Future optimizations are planned
for this functionality.
This removes the use of byte readers from the process that
parses and processes the trace batches.
This modifies the flight recorder to not parse out the trace
clock frequency, since that requires knowledge of the format that's
unfortunate to encode in yet another place. Right now, the trace clock
frequency is considered stable for the lifetime of the program, so just
grab it directly from the runtime.
This change adds an in-band end-of-generation signal to the internal
implementation of runtime.ReadTrace. The internal implementation is
exported via linkname to runtime/trace, so the flight recorder can
identify exactly when a generation has ended. This signal is also useful
for ensuring that subscribers to runtime trace data always see complete
generations, by starting or stopping data streaming only at generation
boundaries.
For #63185
Change-Id: I5c15345981a6bbe9764a3d623448237e983c64ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/673116
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL adds two related features enabled by default via compatibility
GODEBUGs containermaxprocs and updatemaxprocs.
On Linux, containermaxprocs makes the Go runtime consider cgroup CPU
bandwidth limits (quota/period) when setting GOMAXPROCS. If the cgroup
limit is lower than the number of logical CPUs available, then the
cgroup limit takes precedence.
On all OSes, updatemaxprocs makes the Go runtime periodically
recalculate the default GOMAXPROCS value and update GOMAXPROCS if it has
changed. If GOMAXPROCS is set manually, this update does not occur. This
is intended primarily to detect changes to cgroup limits, but it applies
on all OSes because the CPU affinity mask can change as well.
The runtime only considers the limit in the leaf cgroup (the one that
actually contains the process), caching the CPU limit file
descriptor(s), which are periodically reread for updates. This is a
small departure from the original proposed design. It will not consider
limits of parent cgroups (which may be lower than the leaf), and it will
not detection cgroup migration after process start.
We can consider changing this in the future, but the simpler approach is
less invasive; less risk to packages that have some awareness of runtime
internals. e.g., if the runtime periodically opens new files during
execution, file descriptor leak detection is difficult to implement in a
stable way.
For #73193.
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Change-Id: I6a6a636c631c1ae577fb8254960377ba91c5dc98
Reviewed-on: https://go-review.googlesource.com/c/go/+/670497
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
This implementation is zero-alloc when T is a concrete type,
allocates when val contains a method or when T is a interface
and Value was obtained for example through Elem(), in which case
it has to be allocated to avoid sharing the same memory.
goos: linux
goarch: amd64
pkg: reflect
cpu: AMD Ryzen 5 4600G with Radeon Graphics
│ /tmp/bench2 │
│ sec/op │
TypeAssert/TypeAssert[int](int)-12 2.725n ± 1%
TypeAssert/TypeAssert[uint8](int)-12 2.599n ± 1%
TypeAssert/TypeAssert[fmt.Stringer](reflect_test.testTypeWithMethod)-12 8.470n ± 0%
TypeAssert/TypeAssert[fmt.Stringer](*reflect_test.testTypeWithMethod)-12 8.460n ± 1%
TypeAssert/TypeAssert[interface_{}](int)-12 4.181n ± 1%
TypeAssert/TypeAssert[interface_{}](reflect_test.testTypeWithMethod)-12 4.178n ± 1%
TypeAssert/TypeAssert[time.Time](time.Time)-12 2.839n ± 0%
TypeAssert/TypeAssert[func()_string](func()_string)-12 151.1n ± 1%
geomean 6.645n
│ /tmp/bench2 │
│ B/op │
TypeAssert/TypeAssert[int](int)-12 0.000 ± 0%
TypeAssert/TypeAssert[uint8](int)-12 0.000 ± 0%
TypeAssert/TypeAssert[fmt.Stringer](reflect_test.testTypeWithMethod)-12 0.000 ± 0%
TypeAssert/TypeAssert[fmt.Stringer](*reflect_test.testTypeWithMethod)-12 0.000 ± 0%
TypeAssert/TypeAssert[interface_{}](int)-12 0.000 ± 0%
TypeAssert/TypeAssert[interface_{}](reflect_test.testTypeWithMethod)-12 0.000 ± 0%
TypeAssert/TypeAssert[time.Time](time.Time)-12 0.000 ± 0%
TypeAssert/TypeAssert[func()_string](func()_string)-12 72.00 ± 0%
geomean ¹
Fixes#62121
Change-Id: I0911c70c5966672c930d387438643f94a40441c4
GitHub-Last-Rev: ce89a53097
GitHub-Pull-Request: golang/go#71639
Reviewed-on: https://go-review.googlesource.com/c/go/+/648056
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This required adding a new field to SessionState for TLS 1.0–1.2, since
the key exchange is not repeated on resumption. The additional field is
unfortunately not backwards compatible because current Go versions check
that the encoding has no extra data at the end, but will cause
cross-version tickets to be ignored. Relaxed that so we can add fields
in a backwards compatible way the next time.
For the cipher suite, we check that the session's is still acceptable
per the Config. That would arguably make sense here, too: if a Config
for example requires PQ, we should reject resumptions of connections
that didn't use PQ. However, that only applies to pre-TLS 1.3
connections, since in TLS 1.3 we always do a fresh key exchange on
resumption. Since PQ is the only main differentiator between key
exchanges (aside from off-by-default non-PFS RSA, which are controlled
by the cipher suite in TLS 1.0–1.2) and it's PQ-only, we can skip that
check.
Fixes#67516
Change-Id: I6a6a465681a6292edf66c7b8df8f4aba4171a76b
Reviewed-on: https://go-review.googlesource.com/c/go/+/653315
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
This PR adds an helper FileContentDisposition that builds multipart
Content-Disposition header contents with field name and file name,
escaping quotes and escape characters.
The function is then called in the related helper CreateFormFile.
The new function allows users to add other custom MIMEHeaders,
without having to rewrite the char escaping logic of field name and
file name, which is provided by the new helper.
Fixes#46771
Change-Id: Ifc82a79583feb6dd609ca1e6024e612fb58c05ce
GitHub-Last-Rev: 969f846fa9
GitHub-Pull-Request: golang/go#63324
Reviewed-on: https://go-review.googlesource.com/c/go/+/531995
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
This CL adds an enum type, VarKind, that discriminates among
the various kinds of Var, and adds setter/getter methods
for Var's kind field.
Beware: NewVar has a weaker postcondition: the Var objects it
returns are not completely initialized and require a call to
Var.SetKind. This should only affect importers.
No changes are needed to the export data, since the kind can
always be deduced from the context when decoding.
See CL 645656 for the corresponding x/tools changes.
+ test, relnote, API
Updates golang/go#70250
Change-Id: Icde86ad22a880cde6f50bc12bf38004a5c6a1025
Reviewed-on: https://go-review.googlesource.com/c/go/+/645115
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Added implementations for *io/fs.subFS, os.DirFS, and testing/fstest.MapFS.
Amended testing/fstest.TestFS to check behavior.
Addressed TODOs in archive/tar and os.CopyFS around symbolic links.
I am deliberately not changing archive/zip in this CL,
since it currently does not resolve symlinks
as part of its filesystem implementation.
I am unsure of the compatibility restrictions on doing so,
so figured it would be better to address independently.
testing/fstest.MapFS now includes resolution of symlinks,
with MapFile.Data storing the symlink data.
The behavior change there seemed less intrusive,
especially given its intended usage in tests,
and it is especially helpful in testing the io/fs function implementations.
Fixes#49580
Change-Id: I58ec6915e8cc97341cdbfd9c24c67d1b60139447
Reviewed-on: https://go-review.googlesource.com/c/go/+/385534
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Funda Secgin <fundasecgin33@gmail.com>
This updates the new version API for the discussion on #63952.
This change reveals that in fact none of the tests set the
VERSYM_HIDDEN bit. The code before this CL set the hidden flag
for symbols that appear in DynamicVersionNeed, but that is not
an accurate representation of the ELF. The readelf program
does print undefined symbols that way (with a single '@'),
but that doesn't mean that the hidden flag is set.
Leaving tests with the hidden bit set for later.
For #63952
Change-Id: Ida60831e0c9922dfc10f10c7a64bc76a2b197537
Reviewed-on: https://go-review.googlesource.com/c/go/+/635079
Reviewed-by: Austin Clements <austin@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
This makes three related changes that work particularly well together
and would require significant extra work to do separately: it replaces
X25519Kyber768Draft00 with X25519MLKEM768, it makes CurvePreferences
ordering crypto/tls-selected, and applies a preference to PQ key
exchange methods over key shares (to mitigate downgrades).
TestHandshakeServerUnsupportedKeyShare was removed because we are not
rejecting unsupported key shares anymore (nor do we select them, and
rejecting them actively is a MAY). It would have been nice to keep the
test to check we still continue successfully, but testClientHelloFailure
is broken in the face of any server-side behavior which requires writing
any other messages back to the client, or reading them.
Updates #69985Fixes#69393
Change-Id: I58de76f5b8742a9bd4543fd7907c48e038507b19
Reviewed-on: https://go-review.googlesource.com/c/go/+/630775
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
