84 Commits

Author SHA1 Message Date
Robert Findley
0406d3a8e5 go/ast: rename MultiIndexExpr to IndexListExpr
As discussed in #47781, IndexListExpr is one character shorter and has
the advantage of being next to IndexExpr in documentation.

Updates #47781

Change-Id: I709d5c1a79b4f9aebcd6445e4ab0cd6dae45bab7
Reviewed-on: https://go-review.googlesource.com/c/go/+/348609
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-09-08 20:14:03 +00:00
Robert Findley
12eb7331b9 go/ast: rename TParams fields to TypeParams
As discussed in the ast proposal (#47781), there's not really a strong
reason to avoid spelling out 'Type'.

Change-Id: I0ba1bf03b112ea60509a78a89a050a302779d9d0
Reviewed-on: https://go-review.googlesource.com/c/go/+/348375
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-09-08 16:59:04 +00:00
Robert Findley
7637345b6e go/internal/typeparams: remove typeparams.{Get,Set} (cleanup)
These helper functions are no longer necessary, now that type parameters
are enabled; we can access type parameters directly.

When considering the existence or non-existence of type parameters, we
can either check whether node.TParams != nil, or whether
node.TParams.NumFields() > 0. The heuristic I'm using for deciding
between these checks is as follows:
 - For data access, just check node.TParams != nil.
 - For producing errors if type parameters exist, check NumFields() > 0.

Change-Id: I6597536898e975564e9e8bf6a3a91bc798e0f110
Reviewed-on: https://go-review.googlesource.com/c/go/+/346549
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-08-31 21:25:03 +00:00
Rob Findley
5f50a6442e [dev.typeparams] go/internal/typeparams: remove the Enabled guard
Type parameters are now always enabled. Users should guard against type
checking generic code by using the types.Config.GoVersion field.

This cleans up some differences with types2.

Change-Id: Ie3e35a549e456a90a10d6a7e158ff58653cc1394
Reviewed-on: https://go-review.googlesource.com/c/go/+/335033
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-07-16 23:08:28 +00:00
Rob Findley
334f2fc045 [dev.typeparams] go/*: switch from ListExpr to MultiIndexExpr
When instantiating a generic type or function with multiple type
arguments, we need to represent an index expression with multiple
indexes in the AST. Previous to this CL this was done with a new
ast.ListExpr node, which allowed packing multiple expressions into a
single ast.Expr. This compositional pattern can be both inefficient and
cumbersome to work with, and introduces a new node type that only exists
to augment the meaning of an existing node type.

By comparison, other specializations of syntax are given distinct nodes
in go/ast, for example variations of switch or for statements, so the
use of ListExpr was also (arguably) inconsistent.

This CL removes ListExpr, and instead adds a MultiIndexExpr node, which
is exactly like IndexExpr but allows for multiple index arguments. This
requires special handling for this new node type, but a new wrapper in
the typeparams helper package largely mitigates this special handling.

Change-Id: I65eb29c025c599bae37501716284dc7eb953b2ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/327149
Trust: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-07-16 00:44:40 +00:00
Cuong Manh Le
ac2de11cfb [dev.typeparams] all: merge master (37f9a8f) into dev.typeparams
Conflicts:

- src/go/types/check_test.go

  CL 330629 fixed a bug in package qualification logic

- src/internal/buildcfg/exp.go

  CL 329930 make parseExperiments get go arch string as input param

Merge List:

+ 2021-06-25 37f9a8f69d go/types: fix a bug in package qualification logic
+ 2021-06-24 c309c89db5 reflect: document that InterfaceData is a low-entropy RNG
+ 2021-06-24 cce621431a cmd/compile: fix wrong type in SSA generation for OSLICE2ARRPTR
+ 2021-06-24 600a2a4ffb cmd/go: don't try to add replaced versions that won't be selected
+ 2021-06-24 a9bb38222a net: remove hard-coded timeout in dialClosedPort test helper
+ 2021-06-24 86d72fa2cb time: handle invalid UTF-8 byte sequences in quote to prevent panic
+ 2021-06-24 44a12e5f33 cmd/go: search breadth-first instead of depth-first for test dependency cycles
+ 2021-06-24 73496e0df0 net: use absDomainName in the Windows lookupPTR test helper
+ 2021-06-24 222ed1b38a os: enable TestFifoEOF on openbsd
+ 2021-06-22 0ebd5a8de0 cmd/go: update ToolTags based on GOARCH value
+ 2021-06-22 5bd09e5efc spec: unsafe.Add/Slice are not permitted in statement context
+ 2021-06-22 666315b4d3 runtime/internal/atomic: remove incorrect pointer indirection in comment
+ 2021-06-22 63daa774b5 go/types: guard against checking instantiation when generics is disabled
+ 2021-06-22 197a5ee2ab cmd/gofmt: remove stale documentation for the -G flag
+ 2021-06-22 9afd158eb2 go/parser: parse an ast.IndexExpr for a[]
+ 2021-06-21 1bd5a20e3c cmd/go: add a -go flag to 'go mod graph'
+ 2021-06-21 761edf71f6 cmd/internal/moddeps: use a temporary directory for GOMODCACHE if needed
+ 2021-06-21 a0400420ad cmd/internal/moddeps: use -mod=readonly instead of -mod=mod
+ 2021-06-21 3f9ec83b10 cmd/go: document GOPPC64 environment variable
+ 2021-06-21 20bdfba325 go/scanner: fall back to next() when encountering 0 bytes in parseIdentifier
+ 2021-06-21 44f9a3566c database/sql: fix deadlock test in prepare statement

Change-Id: I16490e8ea70ee65081f467223857033842da513a
2021-06-25 11:34:56 +07:00
Rob Findley
9afd158eb2 go/parser: parse an ast.IndexExpr for a[]
To be consistent with Go 1.16, and to preserve as much information in
the AST as possible, parse an ast.IndexExpr with BadExpr Index for the
invalid expression a[].

A go/types test had to be adjusted to account for an additional error
resulting from this change.

We don't have a lot of test coverage for parser error recovery, so
rather than write an ad-hoc test for this issue, add a new go/types test
that checks that the indexed operand is used.

Updates #46403

Change-Id: I21e6ff4179746aaa50e530d4091fded450e69824
Reviewed-on: https://go-review.googlesource.com/c/go/+/329791
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-06-22 02:37:43 +00:00
Rob Findley
54f854fb41 [dev.typeparams] go/parser: accept embedded type literals
This is an approximate port of CL 321109 to go/parser, though go/parser
does not have the same internal APIs as cmd/compile/internal/syntax, so
this CL required some refactoring.

Change-Id: I146ef530c969d61bab99f98f4de94b862e103ddc
Reviewed-on: https://go-review.googlesource.com/c/go/+/325703
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-06-17 02:06:15 +00:00
Rob Findley
ab4b3c4b15 [dev.typeparams] go/parser: accept "~" and "|" interface elements
This is a port of CL 307371 to go/parser, adding support for the new
embedded type expressions. As in that CL, type lists continue to be
accepted.

This CL also revealed a pre-existing bug related to embedded instances:
the parser was failing to parse embedded instances with multiple type
arguments, due to not consuming the initial ','. This is fixed, and
along the way TestErrors is modified to use subtests.

Several missing tests cases were added to exprstring_test.go. These must
have been missed in an earlier CL.

Change-Id: I452769536998cddb1618bebdba675fc09d48a12f
Reviewed-on: https://go-review.googlesource.com/c/go/+/325690
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-06-17 02:06:03 +00:00
Rob Findley
cf73f1a8e4 go/parser: don't parse a nil IndexExpr.Index
When parsing type parameters, an empty type instantiation was parsed as
an IndexExpr with nil Index. This should be considered a breaking change
to parsing: ast.Walk previously assumed that Index was non-nil.

Back out the nil check in ast.Walk, and for now pack an empty argument
list as a non-nil ListExpr with nil Elems.

Alternatives considered:
 - Parsing the entire index expression as a BadExpr: this led to
   inferior errors while type checking.
 - Parsing the Index as a BadExpr: this seems reasonable, but encodes
   strictly less information into the AST.

We may want to opt for one of these alternatives in the future, but for
now let's just fix the breaking change.

Change-Id: I93f2b89641692ac014b8ee98bfa031ed3477afb8
Reviewed-on: https://go-review.googlesource.com/c/go/+/315851
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-05-05 20:58:39 +00:00
kumakichi
291eb0178f go: various minor cleanups with the help of Goland
• fix some typos
    • remove superfluous conversions/parentheses
    • remove superfluous nil checks

Change-Id: I428bf6a7be551b79270567047878c3076dd6f2ff
GitHub-Last-Rev: 3b1c7573cf
GitHub-Pull-Request: golang/go#45799
Reviewed-on: https://go-review.googlesource.com/c/go/+/314069
Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
2021-04-27 18:57:27 +00:00
Rob Findley
6fa12172d9 go/parser: add benchmarks for syntax-only parsing and object resolution
Change-Id: I73d1b2d18ab4051443d66c60df493d1163d0ba3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/306150
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-04-27 15:57:10 +00:00
Yury Smolsky
4f5aec4603 all: remove redundant spaces before . and ,
Change-Id: I6a4bd2544276d0638bddf07ebcf2ee636db30fea
Reviewed-on: https://go-review.googlesource.com/c/go/+/311009
Run-TryBot: Yury Smolsky <yury@smolsky.by>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2021-04-20 00:49:17 +00:00
Rob Findley
02a2ff47ef go/parser: add a SkipObjectResolution mode to bypass object resolution
Parser object resolution is an auxiliary feature in which the parser
attempts to resolve identifiers to their declarations. In functionality,
it significantly overlaps with go/types and in fact cannot be correctly
computed at parse-time without type information (for example, it is
generally not possible to resolve k in the composite lit c{k: v}). Due
to these limitations, it is of limited utility and rarely used.

Now that object resolution is isolated as a post-processing pass, it is
trivial to offer a parser mode that skips it entirely. This CL adds that
mode.

Fixes #45104

Change-Id: I5a2c05437e298964ad2039e1ff98e63d6efbd1af
Reviewed-on: https://go-review.googlesource.com/c/go/+/306149
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-04-16 21:20:31 +00:00
Rob Findley
9e8a312b71 go/parser: move type params in scope for the function signature
Type parameter resolution is a bit tricky: type parameters are in the
function scope, but unlike ordinary parameters may reference eachother.
When resolving the function scope, we must be careful about the order in
which objects are resolved and declared.

Using ordering allows us to avoid passing around temporary scopes for
field declarations.

Add a bunch of tests for this behavior, and skip "_" in resolution tests
as it just adds noise.

For #45221

Change-Id: Id080cddce3fd76396bf86ba5aba856aedf64a458
Reviewed-on: https://go-review.googlesource.com/c/go/+/304456
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-04-16 21:19:23 +00:00
Rob Findley
283f9fdbd3 cmd/dist: add tests using the typeparams build tag
Now that go/ast changes have been guarded behind the typeparams build
tag, we no longer have coverage for tests involving generic code.

Add a new testing step to cmd/dist to run go/... and cmd/gofmt tests
using -tags=typeparams.

Comment out parser object resolution assertions that currently fail, and
which will be fixed by CL 304456.

Fixes #44933

Change-Id: I481dd4246a016f410307865b6c6c2bb3c8e6e3bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/310071
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-04-14 19:32:56 +00:00
Rob Findley
efaf75a216 go/*,cmd/gofmt: guard AST changes with the typeparams build tag
This CL changes our approach to guarding type parameter functionality
and API. Previously, we guarded type parameter functionality with the
parser.parseTypeParams parser mode, and were in the process of hiding
the type parameter API behind the go1.18 build constraint.

These mechanisms had several limitations:
 + Requiring the parser.parseTypeParams mode to be set meant that
   existing tooling would have to opt-in to type parameters in all
   places where it parses Go files.
 + The parseTypeParams mode value had to be copied in several places.
 + go1.18 is not specific to typeparams, making it difficult to set up
   the builders to run typeparams tests.

This CL addresses the above limitations, and completes the task of
hiding the AST API, by switching to a new 'typeparams' build constraint
and adding a new go/internal/typeparams helper package.

The typeparams build constraint is used to conditionally compile the new
AST changes. The typeparams package provides utilities for accessing and
writing the new AST data, so that we don't have to fragment our parser
or type checker logic across build constraints. The typeparams.Enabled
const is used to guard tests that require type parameter support.

The parseTypeParams parser mode is gone, replaced by a new
typeparams.DisableParsing mode with the opposite sense. Now, type
parameters are only parsed if go/parser is compiled with the typeparams
build constraint set AND typeparams.DisableParsing not set. This new
parser mode allows opting out of type parameter parsing for tests.

How exactly to run tests on builders is left to a follow-up CL.

Updates #44933

Change-Id: I3091e42a2e5e2f23e8b2ae584f415a784b9fbd65
Reviewed-on: https://go-review.googlesource.com/c/go/+/300649
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-04-13 22:24:31 +00:00
Rob Findley
4acefa07b1 go/parser: switch to resolving objects as a post-processing pass
Coupling object resolution to parsing complicates the parsing code, and
is a barrier to improvement. It requires passing around context such as
'lhs' or 'keyOk', and even then sometimes requires guess-work, such as
whether to resolve the key in a composite literal.

In this CL we delay object resolution to a separate pass after the file
parse completes. This makes it easier to see logic of scoping, and
removes state from the parsing code. This can enable subsequent
improvements such as optionally skipping object resolution, aligning the
parser with cmd/compile/internal/syntax, and allowing alternative
parsers to reuse object resolution.

The additional AST traversal appears to slow down parsing by around 4%.
That seems small enough not to worry about, especially since performance
sensitive users may eventually be able to disable object resolution
entirely, saving around 18% off the previous baseline. I'll also mail a
speculative CL showing how we can significantly mitigate the cost of
object resolution by transposing scopes.

For #45104

Change-Id: I98d9143fd77ae29e84ec7c3ae2fdb1139510da37
Reviewed-on: https://go-review.googlesource.com/c/go/+/304455
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-03-31 20:13:06 +00:00
Rob Findley
1d8abb3417 go/parser: remove redundant list argument to Parser.shortVarDecl
Change-Id: I75d089a7c1c3cdd50e5d2dafdb3386620efff4c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/304454
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-03-31 02:35:30 +00:00
Rob Findley
152ca79b73 go/parser: add resolution tests for type params
For #45104
For #45221

Change-Id: I8966555f4e8844d5b6766d00d48f7a81868ccf40
Reviewed-on: https://go-review.googlesource.com/c/go/+/304453
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-03-31 02:35:00 +00:00
Rob Findley
c3ec79bca9 go/parser: resolve the type name when parsing a composite lit value
parsePrimaryExpr has to be careful to resolve identifiers used in
composite expressions when parsing in LHS mode. It missed the literal
type name.

Fixes #45136

Change-Id: I3e12f91e3ef5fdb43faa436cdf1240eb3293fe1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/304451
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-03-30 23:55:57 +00:00
Rob Findley
0fdd371e6b go/parser: add data-driven tests for object resolution
Add new tests for object resolution driven by source files with
declarations and uses marked via special comments. This made it easier
to add test coverage while refactoring object resolution for #45104.

Tests are added to a new resolver_test.go file. In a subsequent CL the
resolver.go file will be added, making this choice of file name more
sensible.

For #45104
For #45136
For #45160

Change-Id: I240fccc0de95aa8f2d03e39c77146d4c61f1ef9e
Reviewed-on: https://go-review.googlesource.com/c/go/+/304450
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-03-30 23:55:42 +00:00
Rob Findley
0bd308ff27 go/parser: avoid formatting a panic message if an assertion succeeds
tryResolve is an extremely hot method on the parser. Eliminating this
formatting led to a 20% performance improvement in BenchmarkParse.

Change-Id: Idf8850404bd72d45d1351356427a85086422ea68
Reviewed-on: https://go-review.googlesource.com/c/go/+/302629
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-03-17 17:13:50 +00:00
Rob Findley
a6eeb4add4 go/parser,go/types: hide API changes related to type parameters
While the dev.typeparams branch was merged, the type parameter API is
slated for go1.18. Hide these changes to the go/parser and go/types API.

This was done as follows:
 + For APIs that will probably not be needed for go1.18, simply unexport
   them.
 + For APIs that we expect to export in go1.18, prefix symbols with '_',
   so that the planned (upper-cased) symbol name is apparent.
 + For APIs that must be exported for testing, move both API and tests
   to files guarded by the go1.18 build constraint.
 + parser.ParseTypeParams is unexported and copied wherever it is
   needed.
 + The -G flag is removed from gofmt, replaced by enabling type
   parameters if built with the go1.18 build constraint.

Notably, changes related to type parameters in go/ast are currently left
exported. We're looking at the AST API separately.

The new API diff from 1.16 is:
+pkg go/ast, method (*FuncDecl) IsMethod() bool
+pkg go/ast, method (*ListExpr) End() token.Pos
+pkg go/ast, method (*ListExpr) Pos() token.Pos
+pkg go/ast, type FuncType struct, TParams *FieldList
+pkg go/ast, type ListExpr struct
+pkg go/ast, type ListExpr struct, ElemList []Expr
+pkg go/ast, type TypeSpec struct, TParams *FieldList
+pkg go/types, type Config struct, GoVersion string

Change-Id: I1baf67e26279b49092e774309a836c460979774a
Reviewed-on: https://go-review.googlesource.com/c/go/+/295929
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-03-02 01:50:12 +00:00
Rob Findley
1126bbb82a go/parser: return ast.BadExpr for missing index operands
The parser was returning the indexed operand when a slice or index or
instance expression was missing any index arguments (as in the
expression `a[]`). This can result in returning an *ast.Ident for the
LHS of the (invalid) assignment `a[] = ...` -- in this case parsing the
LHS as just `a`. Unfortunately, as the indexed operand `a` has already
been resolved, this results in a panic for duplicate resolution.

Fix this by instead returning an ast.BadExpr. This can suppress some
subsequent errors from the typechecker, but those errors may or may not
be correct anyway. Other interpretations, such as an *ast.IndexExpr with
bad or missing X, run into potential misinterpretations downstream (both
caused errors in go/types and/or gopls).

Fixes #44504

Change-Id: I5ca8bed4a1861bcc7db8898770b08937110981d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/295151
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-02-22 23:43:15 +00:00
Rob Findley
2f37939a21 go/parser: improve error recovery from invalid selector exprs
Before this CL, the parser consumed the next token following an invalid
selector expr no matter what it was. This leads to poor error recovery
when this next token is a closing delimiter or other reasonable element
of a stop set. As a side-effect, x/tools tests broke when parser logic
for type parameters was introduced, as they threw off the parser
synchronization to the point where the x/tools test bailed out.

This CL introduces a targeted fix that allows the x/tools tests to pass.
More general improvement for parser error recovery should be done for
go1.17.

Change-Id: I44d73d34b6063e62d16a23d24ab7cbce6500239d
Reviewed-on: https://go-review.googlesource.com/c/go/+/293792
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-02-19 01:53:53 +00:00
Rob Findley
5ecb9a7887 [dev.typeparams] go/types: use a new ast.ListExpr for multi-type instances
Modify go/parser to consistently represent type instantiation as an
ast.IndexExpr, rather than use an ast.CallExpr (with Brackets:true) for
instantiations with multiple type parameters. To enable this, introduce
a new ast expr type: ListExpr.

This brings go/types in line with types2, with the exception of a small
change to funcInst to eliminate redundant errors if values are
erroneously used as types. In a subsequent CL, call.go and expr.go will
be marked as reviewed.

This also catches some type instance syntax using '()' that was
previously accepted incorrectly. Tests are updated accordingly.

Change-Id: I30cd0181c7608f1be7486a9a8b63df993b412e85
Reviewed-on: https://go-review.googlesource.com/c/go/+/293010
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-02-18 20:38:41 +00:00
Rob Findley
958927c824 [dev.typeparams] go/parser: error for type instances without ParseTypeParams
It should be an invariant that the parser does not produce ast.CallExprs
with Brackets == true unless parsing with ParseTypeParams.

Fix the one case where this invariant was violated, and add a test for
errors produced in valid generic code when ParseTypeParams is unset. We
did have some coverage of errors in short_test.go, but I find them to be
easier to read in a testdata file and would like to gradually migrate
them there.

Change-Id: If2d174377087daa1b820cabc2b5bf8bcb0b39d8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/284192
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>
2021-01-19 21:28:51 +00:00
Robert Griesemer
91803a2df3 [dev.typeparams] merge: merge branch 'dev.regabi' into 'dev.typeparams'
The following files had merge conflicts and were merged manually:

	src/cmd/compile/fmtmap_test.go
	src/cmd/compile/internal/gc/noder.go
	src/go/parser/error_test.go
	test/assign.go
	test/chan/perm.go
	test/fixedbugs/issue22822.go
	test/fixedbugs/issue4458.go
	test/init.go
	test/interface/explicit.go
	test/map1.go
	test/method2.go

The following files had manual changes to make tests pass:

	test/run.go
	test/used.go
	src/cmd/compile/internal/types2/stdlib_test.go

Change-Id: Ia495aaaa80ce321ee4ec2a9105780fbe913dbd4c
2020-12-14 12:13:36 -08:00
Russ Cox
f1980efb92 all: update to use os.ReadDir where appropriate
os.ReadDir is a replacement for ioutil.ReadDir that returns
a slice of fs.DirEntry instead of fs.FileInfo, meaning it is the
more efficient form.

This CL updates call sites throughout the Go source tree
wherever possible. As usual, code built using the Go 1.4
bootstrap toolchain is not included. There is also a use in
go/build that appears in the public API and can't be changed,
at least not without additional changes.

Fixes #42026.

Change-Id: Icfc9dd52c6045020f6830e22c72128499462d561
Reviewed-on: https://go-review.googlesource.com/c/go/+/266366
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-12-09 19:12:27 +00:00
Russ Cox
4f1b0a44cb all: update to use os.ReadFile, os.WriteFile, os.CreateTemp, os.MkdirTemp
As part of #42026, these helpers from io/ioutil were moved to os.
(ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.)

Update the Go tree to use the preferred names.

As usual, code compiled with the Go 1.4 bootstrap toolchain
and code vendored from other sources is excluded.

ReadDir changes are in a separate CL, because they are not a
simple search and replace.

For #42026.

Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/266365
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-12-09 19:12:23 +00:00
Rob Findley
6015c4e543 [dev.typeparams] go/*: add TODOs from CLs importing dev.go2go changes
With the plurality of CLs importing dev.go2go changes it's getting hard
to track all of the code review comments that were deferred for later
consideration. Add some TODOs to capture these comments in the source,
so that they may be more easily located.

Change-Id: I5caf085fec11ca8992b7affe6feb0a7aa202f21f
Reviewed-on: https://go-review.googlesource.com/c/go/+/276254
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-12-08 22:00:26 +00:00
Carlos Alexandro Becker
48838c35dc go/parser: ignore subdirectories in ParseDir
Issue and PR on GoReleaser:
- https://github.com/goreleaser/goreleaser/issues/1897
- https://github.com/goreleaser/goreleaser/pull/1899

Fixes #42951.

Change-Id: Ia0d6018e0bad59cd60cd600188c368c431032a4b
GitHub-Last-Rev: be59d85fe2
GitHub-Pull-Request: golang/go#42581
Reviewed-on: https://go-review.googlesource.com/c/go/+/269897
Trust: Robert Griesemer <gri@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-12-02 23:05:56 +00:00
Rob Findley
0fb733b7f7 [dev.typeparams] go/parser: support the ParseTypeParams mode
Support is added for parsing type parameters only if the ParseTypeParams
mode is set, otherwise emitting syntax errors for source code that is
invalid without type parameters.

Rather than have large conditional blocks switching between legacy
parser logic and new parser logic, effort is made to minimize special
handling for ParseTypeParams.

Change-Id: I243f6c4b9b8eb1313b838e8649b6cc1e5e8339ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/271218
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>
2020-11-24 19:44:27 +00:00
Rob Findley
b56762129e [dev.typeparams] import go2go changes to parse type parameters
This CL imports changes on the go2go branch to support parsing type
params, as well as the unsubmitted changes from CL 269300 to remove
support for parenthesize type parameter syntax.

Change-Id: I27ab942ce69eab62c2a1800f8f9661c4dcb233fe
Reviewed-on: https://go-review.googlesource.com/c/go/+/270857
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>
2020-11-24 19:44:06 +00:00
Russ Cox
1b09d43067 all: update references to symbols moved from io/ioutil to io
The old ioutil references are still valid, but update our code
to reflect best practices and get used to the new locations.

Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.
Also excluded vendored code.

For #41190.

Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/263142
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-20 18:41:18 +00:00
Russ Cox
7bb721b938 all: update references to symbols moved from os to io/fs
The old os references are still valid, but update our code
to reflect best practices and get used to the new locations.

Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.

For #41190.

Change-Id: I8f9526977867c10a221e2f392f78d7dec073f1bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/243907
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-10-20 02:32:42 +00:00
Ivan Trubach
585e31df63 cmd/doc: fix merging comments in -src mode
These changes fix go doc -src mode that vomits comments from random files if
filesystem does not sort files by name. The issue was with parse.ParseDir
using the Readdir order of files, which varies between platforms and filesystem
implementations. Another option is to merge comments using token.FileSet.Iterate
order in cmd/doc, but since ParseDir is mostly used in go doc, I’ve opted for
smaller change because it’s unlikely to break other uses or cause any perfomance
issues.

Example (macOS APFS): `go doc -src net.ListenPacket`

Change-Id: I7f9f368c7d9ccd9a2cbc48665f2cb9798c7b3a3f
GitHub-Last-Rev: 654fb45042
GitHub-Pull-Request: golang/go#36104
Reviewed-on: https://go-review.googlesource.com/c/go/+/210999
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-03-05 03:55:11 +00:00
Robert Griesemer
07b72d9189 go/parser: use valid position when reporting an error (copy-paste bug)
This is a follow-up on https://golang.org/cl/202581.

Updates #33649.

Change-Id: Ib078fed983792c5493bdbed6d33e21b86856894a
Reviewed-on: https://go-review.googlesource.com/c/go/+/204041
Run-TryBot: Robert Griesemer <gri@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
2019-10-29 19:20:52 +00:00
Robert Griesemer
3e457030d9 go/parser, go/ast: correctly take into account presence of } in block
Correctly track whether the closing } of a block (or a function body)
is present or not in the AST and report correct End() positions in
each case.

There are more cases like this but this CL addresses an immediate
issue and sets a precedent for how to fix similar cases if a need
arises.

Fixes #33649.

Change-Id: Id6662ddaac09f3c15f8003edc9275fe2b0c41c78
Reviewed-on: https://go-review.googlesource.com/c/go/+/202581
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
2019-10-28 20:47:07 +00:00
ajz01
57ad6ef15d go/parser: remove superfluous case from switch statement
Change-Id: I96a9b34bf8e42c21a3e0258cbc8b1416328834be
GitHub-Last-Rev: 32709619a6
GitHub-Pull-Request: golang/go#35066
Reviewed-on: https://go-review.googlesource.com/c/go/+/202598
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-10-22 04:05:25 +00:00
Robert Griesemer
a59808ed01 go/parser: better error (recovery) for Allman/BSD-style func decls
This matches the behavior and error of cmd/compile.

Fixes #34946.

Change-Id: I329ef358deea63d8425f76f1d54c95749b96c365
Reviewed-on: https://go-review.googlesource.com/c/go/+/202484
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-21 23:36:02 +00:00
Robert Griesemer
99aa56a437 go/parser: return partial result from ParseExpr in case of error
Remove redundant code and improve documentation in the process.

Fixes #34211.

Change-Id: I9a6d1467f1a2c98a163f41f9df147fc6500c6fad
Reviewed-on: https://go-review.googlesource.com/c/go/+/196077
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-09-18 04:47:40 +00:00
Ben Shi
d12c62d12d go/parser: fix ignored errors in ParseExprFrom
This CL fixes a bug in ParseExprFrom which makes
error messages ignored when there are 10+ errors
in a single expression.

fixes #34241
fixes #34274

Change-Id: I29a82d3e3e726279005eb6fbcd7ee3aebffaa679
Reviewed-on: https://go-review.googlesource.com/c/go/+/194638
Run-TryBot: Ben Shi <powerman1st@163.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-09-15 00:18:30 +00:00
Agniva De Sarker
a19c0ceda5 Revert "go/parser: include more comments in a struct or interface"
This reverts commit https://golang.org/cl/161177/.

Reason for revert: this led to non-contiguous comments spaced
by an empty line to be grouped into a single CommentGroup

Fixes #32944
Updates #10858

Change-Id: I5e16663b308c3b560496da8e66c33befdf9ed9dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/185040
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-07-08 20:59:49 +00:00
Agniva De Sarker
43e8fd4ef1 go/parser: include more comments in a struct or interface
While parsing inside a struct or an interface, skipping over empty lines
too to collect the next group of comments. We do not need to skip
over more than 1 empty line since gofmt already removes multiple
empty consecutive lines.

Fixes #10858

Change-Id: I0c97b65b5fc44e225e5dc7871ace24f43419ce08
Reviewed-on: https://go-review.googlesource.com/c/go/+/161177
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-03-06 06:05:15 +00:00
Igor Zhilianin
f90e89e675 all: fix a bunch of misspellings
Change-Id: If2954bdfc551515403706b2cd0dde94e45936e08
GitHub-Last-Rev: d4cfc41a55
GitHub-Pull-Request: golang/go#28049
Reviewed-on: https://go-review.googlesource.com/c/140299
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-10-06 15:40:03 +00:00
Robert Griesemer
db37e1600f go/parser: make sure we have a valid AST when 'if' condition is missing
This prevents a crash in go/types due to a nil condition in an 'if'
statement. There's more we can do to make go/types more robust but
this will address the immediate cause and also makes sure that the
parser returns a valid AST in this case.

Fixes #25438.

Change-Id: Ie55dc2c722352a5ecb17af6a16983741e8a8b515
Reviewed-on: https://go-review.googlesource.com/113735
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Dominik Honnef <dominik@honnef.co>
Reviewed-by: Alan Donovan <adonovan@google.com>
2018-05-18 16:01:32 +00:00
Robert Griesemer
7ac393a3f2 go/parser: improved error message for unexpected literals
R=go1.11

This is a follow up for #11377 which reported that an error like

/tmp/xx.go:9:6: expected '(', found 'IDENT' F1

shouldn't print 'IDENT', as it's just an internal detail.
The relevant change wasn't made in the original fix, so here it is.

For #11377.

Change-Id: Ib76957d86b88e3e63646fbe4abf03a3b9d045139
Reviewed-on: https://go-review.googlesource.com/87900
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-02-12 21:43:41 +00:00
Robert Griesemer
5408d79950 go/parser: remove newly introduced TODO (cleanup)
R=go1.11

No semantic change.

For #23434.

Change-Id: Iafdb062b0ebe6cd6e51f9a98b62b1d10f1bacc5c
Reviewed-on: https://go-review.googlesource.com/87899
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-02-12 21:43:34 +00:00