This allows clients to efficiently reconstruct the original input in the case of ErrBufferExceeded. TestMaxBufferReconstruction now properly verifies this.
R=bradfitz
CC=golang-codereviews
https://golang.org/cl/47770043
Two tweaks enable this:
1) Updating the raw and data span pointers when Tokenizer.Next is called, even
if an error has occurred. This prevents duplicate data from being returned by
Raw in the common case of an EOF.
2) Treating '</>' as an empty comment token to expose the raw text as a
tokenization event. (This matches the semantics of other non-token events,
e.g., '</ >' is treated as '<!-- -->'.)
Fixesgolang/go#7029.
R=golang-codereviews, r, bradfitz
CC=golang-codereviews
https://golang.org/cl/46370043
NewReader is a convenience function for finding the encoding of
an io.Reader and making a UTF-8 version of that Reader.
R=nigeltao
CC=golang-dev
https://golang.org/cl/43510043
Lookup now returns the canonical name as well as the Encoding.
This will make it easier for users to discover what encoding they
actually have as a return value from functions in this package.
They will also be able to store the name for re-use.
R=nigeltao, mpvl
CC=golang-dev
https://golang.org/cl/30090043
either (0, nil) or an (n, err) such that n > 0 && err != nil. Both
cases are valid by the io.Reader contract.
R=r
CC=golang-dev
https://golang.org/cl/12513043
