rpc: implement full bi-directional communication (#18471)

New APIs added:

    client.RegisterName(namespace, service) // makes service available to server
    client.Notify(ctx, method, args...)     // sends a notification
    ClientFromContext(ctx)                  // to get a client in handler method

This is essentially a rewrite of the server-side code. JSON-RPC
processing code is now the same on both server and client side. Many
minor issues were fixed in the process and there is a new test suite for
JSON-RPC spec compliance (and non-compliance in some cases).

List of behavior changes:

- Method handlers are now called with a per-request context instead of a
  per-connection context. The context is canceled right after the method
  returns.
- Subscription error channels are always closed when the connection
  ends. There is no need to also wait on the Notifier's Closed channel
  to detect whether the subscription has ended.
- Client now omits "params" instead of sending "params": null when there
  are no arguments to a call. The previous behavior was not compliant
  with the spec. The server still accepts "params": null.
- Floating point numbers are allowed as "id". The spec doesn't allow
  them, but we handle request "id" as json.RawMessage and guarantee that
  the same number will be sent back.
- Logging is improved significantly. There is now a message at DEBUG
  level for each RPC call served.

rpc/doc.go

@@ -15,43 +15,49 @@
 // along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
 
 /*
-Package rpc provides access to the exported methods of an object across a network
-or other I/O connection. After creating a server instance objects can be registered,
-making it visible from the outside. Exported methods that follow specific
-conventions can be called remotely. It also has support for the publish/subscribe
-pattern.
+
+Package rpc implements bi-directional JSON-RPC 2.0 on multiple transports.
+
+It provides access to the exported methods of an object across a network or other I/O
+connection. After creating a server or client instance, objects can be registered to make
+them visible as 'services'. Exported methods that follow specific conventions can be
+called remotely. It also has support for the publish/subscribe pattern.
+
+RPC Methods
 
 Methods that satisfy the following criteria are made available for remote access:
- - object must be exported
+
  - method must be exported
  - method returns 0, 1 (response or error) or 2 (response and error) values
  - method argument(s) must be exported or builtin types
  - method returned value(s) must be exported or builtin types
 
 An example method:
+
  func (s *CalcService) Add(a, b int) (int, error)
 
-When the returned error isn't nil the returned integer is ignored and the error is
-sent back to the client. Otherwise the returned integer is sent back to the client.
+When the returned error isn't nil the returned integer is ignored and the error is sent
+back to the client. Otherwise the returned integer is sent back to the client.
 
-Optional arguments are supported by accepting pointer values as arguments. E.g.
-if we want to do the addition in an optional finite field we can accept a mod
-argument as pointer value.
+Optional arguments are supported by accepting pointer values as arguments. E.g. if we want
+to do the addition in an optional finite field we can accept a mod argument as pointer
+value.
 
- func (s *CalService) Add(a, b int, mod *int) (int, error)
+ func (s *CalcService) Add(a, b int, mod *int) (int, error)
 
-This RPC method can be called with 2 integers and a null value as third argument.
-In that case the mod argument will be nil. Or it can be called with 3 integers,
-in that case mod will be pointing to the given third argument. Since the optional
-argument is the last argument the RPC package will also accept 2 integers as
-arguments. It will pass the mod argument as nil to the RPC method.
+This RPC method can be called with 2 integers and a null value as third argument. In that
+case the mod argument will be nil. Or it can be called with 3 integers, in that case mod
+will be pointing to the given third argument. Since the optional argument is the last
+argument the RPC package will also accept 2 integers as arguments. It will pass the mod
+argument as nil to the RPC method.
 
-The server offers the ServeCodec method which accepts a ServerCodec instance. It will
-read requests from the codec, process the request and sends the response back to the
-client using the codec. The server can execute requests concurrently. Responses
-can be sent back to the client out of order.
+The server offers the ServeCodec method which accepts a ServerCodec instance. It will read
+requests from the codec, process the request and sends the response back to the client
+using the codec. The server can execute requests concurrently. Responses can be sent back
+to the client out of order.
 
 An example server which uses the JSON codec:
+
  type CalculatorService struct {}
 
  func (s *CalculatorService) Add(a, b int) int {
@@ -73,26 +79,40 @@ An example server which uses the JSON codec:
  for {
 	c, _ := l.AcceptUnix()
 	codec := v2.NewJSONCodec(c)
-	go server.ServeCodec(codec)
+	go server.ServeCodec(codec, 0)
  }
 
+Subscriptions
+
 The package also supports the publish subscribe pattern through the use of subscriptions.
-A method that is considered eligible for notifications must satisfy the following criteria:
- - object must be exported
+A method that is considered eligible for notifications must satisfy the following
+criteria:
+
  - method must be exported
  - first method argument type must be context.Context
  - method argument(s) must be exported or builtin types
- - method must return the tuple Subscription, error
+ - method must have return types (rpc.Subscription, error)
 
 An example method:
- func (s *BlockChainService) NewBlocks(ctx context.Context) (Subscription, error) {
+
+ func (s *BlockChainService) NewBlocks(ctx context.Context) (rpc.Subscription, error) {
  	...
  }
 
-Subscriptions are deleted when:
- - the user sends an unsubscribe request
- - the connection which was used to create the subscription is closed. This can be initiated
-   by the client and server. The server will close the connection on a write error or when
-   the queue of buffered notifications gets too big.
+When the service containing the subscription method is registered to the server, for
+example under the "blockchain" namespace, a subscription is created by calling the
+"blockchain_subscribe" method.
+
+Subscriptions are deleted when the user sends an unsubscribe request or when the
+connection which was used to create the subscription is closed. This can be initiated by
+the client and server. The server will close the connection for any write error.
+
+For more information about subscriptions, see https://github.com/ethereum/go-ethereum/wiki/RPC-PUB-SUB.
+
+Reverse Calls
+
+In any method handler, an instance of rpc.Client can be accessed through the
+ClientFromContext method. Using this client instance, server-to-client method calls can be
+performed on the RPC connection.
 */
 package rpc

rpc/errors.go

@@ -18,18 +18,31 @@ package rpc
 
 import "fmt"
 
-// request is for an unknown service
-type methodNotFoundError struct {
-	service string
-	method  string
-}
+const defaultErrorCode = -32000
+
+type methodNotFoundError struct{ method string }
 
 func (e *methodNotFoundError) ErrorCode() int { return -32601 }
 
 func (e *methodNotFoundError) Error() string {
-	return fmt.Sprintf("The method %s%s%s does not exist/is not available", e.service, serviceMethodSeparator, e.method)
+	return fmt.Sprintf("the method %s does not exist/is not available", e.method)
+}
+
+type subscriptionNotFoundError struct{ namespace, subscription string }
+
+func (e *subscriptionNotFoundError) ErrorCode() int { return -32601 }
+
+func (e *subscriptionNotFoundError) Error() string {
+	return fmt.Sprintf("no %q subscription in %s namespace", e.subscription, e.namespace)
 }
 
+// Invalid JSON was received by the server.
+type parseError struct{ message string }
+
+func (e *parseError) ErrorCode() int { return -32700 }
+
+func (e *parseError) Error() string { return e.message }
+
 // received message isn't a valid request
 type invalidRequestError struct{ message string }
 
@@ -50,17 +63,3 @@ type invalidParamsError struct{ message string }
 func (e *invalidParamsError) ErrorCode() int { return -32602 }
 
 func (e *invalidParamsError) Error() string { return e.message }
-
-// logic error, callback returned an error
-type callbackError struct{ message string }
-
-func (e *callbackError) ErrorCode() int { return -32000 }
-
-func (e *callbackError) Error() string { return e.message }
-
-// issued when a request is received after the server is issued to stop.
-type shutdownError struct{}
-
-func (e *shutdownError) ErrorCode() int { return -32000 }
-
-func (e *shutdownError) Error() string { return "server is shutting down" }

rpc/http.go

@@ -37,38 +37,39 @@ import (
 
 const (
 	maxRequestContentLength = 1024 * 512
+	contentType             = "application/json"
 )
 
-var (
-	// https://www.jsonrpc.org/historical/json-rpc-over-http.html#id13
-	acceptedContentTypes = []string{"application/json", "application/json-rpc", "application/jsonrequest"}
-	contentType          = acceptedContentTypes[0]
-	nullAddr, _          = net.ResolveTCPAddr("tcp", "127.0.0.1:0")
-)
+// https://www.jsonrpc.org/historical/json-rpc-over-http.html#id13
+var acceptedContentTypes = []string{contentType, "application/json-rpc", "application/jsonrequest"}
 
 type httpConn struct {
 	client    *http.Client
 	req       *http.Request
 	closeOnce sync.Once
-	closed    chan struct{}
+	closed    chan interface{}
 }
 
 // httpConn is treated specially by Client.
-func (hc *httpConn) LocalAddr() net.Addr              { return nullAddr }
-func (hc *httpConn) RemoteAddr() net.Addr             { return nullAddr }
-func (hc *httpConn) SetReadDeadline(time.Time) error  { return nil }
-func (hc *httpConn) SetWriteDeadline(time.Time) error { return nil }
-func (hc *httpConn) SetDeadline(time.Time) error      { return nil }
-func (hc *httpConn) Write([]byte) (int, error)        { panic("Write called") }
-
-func (hc *httpConn) Read(b []byte) (int, error) {
+func (hc *httpConn) Write(context.Context, interface{}) error {
+	panic("Write called on httpConn")
+}
+
+func (hc *httpConn) RemoteAddr() string {
+	return hc.req.URL.String()
+}
+
+func (hc *httpConn) Read() ([]*jsonrpcMessage, bool, error) {
 	<-hc.closed
-	return 0, io.EOF
+	return nil, false, io.EOF
 }
 
-func (hc *httpConn) Close() error {
+func (hc *httpConn) Close() {
 	hc.closeOnce.Do(func() { close(hc.closed) })
-	return nil
+}
+
+func (hc *httpConn) Closed() <-chan interface{} {
+	return hc.closed
 }
 
 // HTTPTimeouts represents the configuration params for the HTTP RPC server.
@@ -114,8 +115,8 @@ func DialHTTPWithClient(endpoint string, client *http.Client) (*Client, error) {
 	req.Header.Set("Accept", contentType)
 
 	initctx := context.Background()
-	return newClient(initctx, func(context.Context) (net.Conn, error) {
-		return &httpConn{client: client, req: req, closed: make(chan struct{})}, nil
+	return newClient(initctx, func(context.Context) (ServerCodec, error) {
+		return &httpConn{client: client, req: req, closed: make(chan interface{})}, nil
 	})
 }
 
@@ -184,17 +185,30 @@ func (hc *httpConn) doRequest(ctx context.Context, msg interface{}) (io.ReadClos
 	return resp.Body, nil
 }
 
-// httpReadWriteNopCloser wraps a io.Reader and io.Writer with a NOP Close method.
-type httpReadWriteNopCloser struct {
+// httpServerConn turns a HTTP connection into a Conn.
+type httpServerConn struct {
 	io.Reader
 	io.Writer
+	r *http.Request
 }
 
-// Close does nothing and returns always nil
-func (t *httpReadWriteNopCloser) Close() error {
-	return nil
+func newHTTPServerConn(r *http.Request, w http.ResponseWriter) ServerCodec {
+	body := io.LimitReader(r.Body, maxRequestContentLength)
+	conn := &httpServerConn{Reader: body, Writer: w, r: r}
+	return NewJSONCodec(conn)
 }
 
+// Close does nothing and always returns nil.
+func (t *httpServerConn) Close() error { return nil }
+
+// RemoteAddr returns the peer address of the underlying connection.
+func (t *httpServerConn) RemoteAddr() string {
+	return t.r.RemoteAddr
+}
+
+// SetWriteDeadline does nothing and always returns nil.
+func (t *httpServerConn) SetWriteDeadline(time.Time) error { return nil }
+
 // NewHTTPServer creates a new HTTP RPC server around an API provider.
 //
 // Deprecated: Server implements http.Handler
@@ -226,7 +240,7 @@ func NewHTTPServer(cors []string, vhosts []string, timeouts HTTPTimeouts, srv ht
 }
 
 // ServeHTTP serves JSON-RPC requests over HTTP.
-func (srv *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 	// Permit dumb empty requests for remote health-checks (AWS)
 	if r.Method == http.MethodGet && r.ContentLength == 0 && r.URL.RawQuery == "" {
 		return
@@ -249,12 +263,10 @@ func (srv *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 		ctx = context.WithValue(ctx, "Origin", origin)
 	}
 
-	body := io.LimitReader(r.Body, maxRequestContentLength)
-	codec := NewJSONCodec(&httpReadWriteNopCloser{body, w})
-	defer codec.Close()
-
 	w.Header().Set("content-type", contentType)
-	srv.ServeSingleRequest(ctx, codec, OptionMethodInvocation)
+	codec := newHTTPServerConn(r, w)
+	defer codec.Close()
+	s.serveSingleRequest(ctx, codec)
 }
 
 // validateRequest returns a non-zero response code and error message if the

rpc/inproc.go

@@ -24,10 +24,10 @@ import (
 // DialInProc attaches an in-process connection to the given RPC server.
 func DialInProc(handler *Server) *Client {
 	initctx := context.Background()
-	c, _ := newClient(initctx, func(context.Context) (net.Conn, error) {
+	c, _ := newClient(initctx, func(context.Context) (ServerCodec, error) {
 		p1, p2 := net.Pipe()
 		go handler.ServeCodec(NewJSONCodec(p1), OptionMethodInvocation|OptionSubscriptions)
-		return p2, nil
+		return NewJSONCodec(p2), nil
 	})
 	return c
 }

rpc/ipc.go

@@ -25,17 +25,17 @@ import (
 )
 
 // ServeListener accepts connections on l, serving JSON-RPC on them.
-func (srv *Server) ServeListener(l net.Listener) error {
+func (s *Server) ServeListener(l net.Listener) error {
 	for {
 		conn, err := l.Accept()
 		if netutil.IsTemporaryError(err) {
-			log.Warn("IPC accept error", "err", err)
+			log.Warn("RPC accept error", "err", err)
 			continue
 		} else if err != nil {
 			return err
 		}
-		log.Trace("IPC accepted connection")
-		go srv.ServeCodec(NewJSONCodec(conn), OptionMethodInvocation|OptionSubscriptions)
+		log.Trace("Accepted RPC connection", "conn", conn.RemoteAddr())
+		go s.ServeCodec(NewJSONCodec(conn), OptionMethodInvocation|OptionSubscriptions)
 	}
 }
 
@@ -46,7 +46,11 @@ func (srv *Server) ServeListener(l net.Listener) error {
 // The context is used for the initial connection establishment. It does not
 // affect subsequent interactions with the client.
 func DialIPC(ctx context.Context, endpoint string) (*Client, error) {
-	return newClient(ctx, func(ctx context.Context) (net.Conn, error) {
-		return newIPCConnection(ctx, endpoint)
+	return newClient(ctx, func(ctx context.Context) (ServerCodec, error) {
+		conn, err := newIPCConnection(ctx, endpoint)
+		if err != nil {
+			return nil, err
+		}
+		return NewJSONCodec(conn), err
 	})
 }

rpc/json_test.go

-// Copyright 2015 The go-ethereum Authors
-// This file is part of the go-ethereum library.
-//
-// The go-ethereum library is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Lesser General Public License as published by
-// the Free Software Foundation, either version 3 of the License, or
-// (at your option) any later version.
-//
-// The go-ethereum library is distributed in the hope that it will be useful,
-// but WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-// GNU Lesser General Public License for more details.
-//
-// You should have received a copy of the GNU Lesser General Public License
-// along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
-
-package rpc
-
-import (
-	"bufio"
-	"bytes"
-	"encoding/json"
-	"reflect"
-	"strconv"
-	"testing"
-)
-
-type RWC struct {
-	*bufio.ReadWriter
-}
-
-func (rwc *RWC) Close() error {
-	return nil
-}
-
-func TestJSONRequestParsing(t *testing.T) {
-	server := NewServer()
-	service := new(Service)
-
-	if err := server.RegisterName("calc", service); err != nil {
-		t.Fatalf("%v", err)
-	}
-
-	req := bytes.NewBufferString(`{"id": 1234, "jsonrpc": "2.0", "method": "calc_add", "params": [11, 22]}`)
-	var str string
-	reply := bytes.NewBufferString(str)
-	rw := &RWC{bufio.NewReadWriter(bufio.NewReader(req), bufio.NewWriter(reply))}
-
-	codec := NewJSONCodec(rw)
-
-	requests, batch, err := codec.ReadRequestHeaders()
-	if err != nil {
-		t.Fatalf("%v", err)
-	}
-
-	if batch {
-		t.Fatalf("Request isn't a batch")
-	}
-
-	if len(requests) != 1 {
-		t.Fatalf("Expected 1 request but got %d requests - %v", len(requests), requests)
-	}
-
-	if requests[0].service != "calc" {
-		t.Fatalf("Expected service 'calc' but got '%s'", requests[0].service)
-	}
-
-	if requests[0].method != "add" {
-		t.Fatalf("Expected method 'Add' but got '%s'", requests[0].method)
-	}
-
-	if rawId, ok := requests[0].id.(*json.RawMessage); ok {
-		id, e := strconv.ParseInt(string(*rawId), 0, 64)
-		if e != nil {
-			t.Fatalf("%v", e)
-		}
-		if id != 1234 {
-			t.Fatalf("Expected id 1234 but got %d", id)
-		}
-	} else {
-		t.Fatalf("invalid request, expected *json.RawMesage got %T", requests[0].id)
-	}
-
-	var arg int
-	args := []reflect.Type{reflect.TypeOf(arg), reflect.TypeOf(arg)}
-
-	v, err := codec.ParseRequestArguments(args, requests[0].params)
-	if err != nil {
-		t.Fatalf("%v", err)
-	}
-
-	if len(v) != 2 {
-		t.Fatalf("Expected 2 argument values, got %d", len(v))
-	}
-
-	if v[0].Int() != 11 || v[1].Int() != 22 {
-		t.Fatalf("expected %d == 11 && %d == 22", v[0].Int(), v[1].Int())
-	}
-}
-
-func TestJSONRequestParamsParsing(t *testing.T) {
-
-	var (
-		stringT = reflect.TypeOf("")
-		intT    = reflect.TypeOf(0)
-		intPtrT = reflect.TypeOf(new(int))
-
-		stringV = reflect.ValueOf("abc")
-		i       = 1
-		intV    = reflect.ValueOf(i)
-		intPtrV = reflect.ValueOf(&i)
-	)
-
-	var validTests = []struct {
-		input    string
-		argTypes []reflect.Type
-		expected []reflect.Value
-	}{
-		{`[]`, []reflect.Type{}, []reflect.Value{}},
-		{`[]`, []reflect.Type{intPtrT}, []reflect.Value{intPtrV}},
-		{`[1]`, []reflect.Type{intT}, []reflect.Value{intV}},
-		{`[1,"abc"]`, []reflect.Type{intT, stringT}, []reflect.Value{intV, stringV}},
-		{`[null]`, []reflect.Type{intPtrT}, []reflect.Value{intPtrV}},
-		{`[null,"abc"]`, []reflect.Type{intPtrT, stringT, intPtrT}, []reflect.Value{intPtrV, stringV, intPtrV}},
-		{`[null,"abc",null]`, []reflect.Type{intPtrT, stringT, intPtrT}, []reflect.Value{intPtrV, stringV, intPtrV}},
-	}
-
-	codec := jsonCodec{}
-
-	for _, test := range validTests {
-		params := (json.RawMessage)([]byte(test.input))
-		args, err := codec.ParseRequestArguments(test.argTypes, params)
-
-		if err != nil {
-			t.Fatal(err)
-		}
-
-		var match []interface{}
-		json.Unmarshal([]byte(test.input), &match)
-
-		if len(args) != len(test.argTypes) {
-			t.Fatalf("expected %d parsed args, got %d", len(test.argTypes), len(args))
-		}
-
-		for i, arg := range args {
-			expected := test.expected[i]
-
-			if arg.Kind() != expected.Kind() {
-				t.Errorf("expected type for param %d in %s", i, test.input)
-			}
-
-			if arg.Kind() == reflect.Int && arg.Int() != expected.Int() {
-				t.Errorf("expected int(%d), got int(%d) in %s", expected.Int(), arg.Int(), test.input)
-			}
-
-			if arg.Kind() == reflect.String && arg.String() != expected.String() {
-				t.Errorf("expected string(%s), got string(%s) in %s", expected.String(), arg.String(), test.input)
-			}
-		}
-	}
-
-	var invalidTests = []struct {
-		input    string
-		argTypes []reflect.Type
-	}{
-		{`[]`, []reflect.Type{intT}},
-		{`[null]`, []reflect.Type{intT}},
-		{`[1]`, []reflect.Type{stringT}},
-		{`[1,2]`, []reflect.Type{stringT}},
-		{`["abc", null]`, []reflect.Type{stringT, intT}},
-	}
-
-	for i, test := range invalidTests {
-		if _, err := codec.ParseRequestArguments(test.argTypes, test.input); err == nil {
-			t.Errorf("expected test %d - %s to fail", i, test.input)
-		}
-	}
-}

rpc/server_test.go

@@ -17,146 +17,136 @@
 package rpc
 
 import (
-	"context"
-	"encoding/json"
+	"bufio"
+	"bytes"
+	"io"
+	"io/ioutil"
 	"net"
-	"reflect"
+	"path/filepath"
+	"strings"
 	"testing"
 	"time"
 )
 
-type Service struct{}
-
-type Args struct {
-	S string
-}
-
-func (s *Service) NoArgsRets() {
-}
-
-type Result struct {
-	String string
-	Int    int
-	Args   *Args
-}
-
-func (s *Service) Echo(str string, i int, args *Args) Result {
-	return Result{str, i, args}
-}
-
-func (s *Service) EchoWithCtx(ctx context.Context, str string, i int, args *Args) Result {
-	return Result{str, i, args}
-}
-
-func (s *Service) Sleep(ctx context.Context, duration time.Duration) {
-	select {
-	case <-time.After(duration):
-	case <-ctx.Done():
-	}
-}
-
-func (s *Service) Rets() (string, error) {
-	return "", nil
-}
-
-func (s *Service) InvalidRets1() (error, string) {
-	return nil, ""
-}
-
-func (s *Service) InvalidRets2() (string, string) {
-	return "", ""
-}
-
-func (s *Service) InvalidRets3() (string, string, error) {
-	return "", "", nil
-}
-
-func (s *Service) Subscription(ctx context.Context) (*Subscription, error) {
-	return nil, nil
-}
-
 func TestServerRegisterName(t *testing.T) {
 	server := NewServer()
-	service := new(Service)
+	service := new(testService)
 
-	if err := server.RegisterName("calc", service); err != nil {
+	if err := server.RegisterName("test", service); err != nil {
 		t.Fatalf("%v", err)
 	}
 
-	if len(server.services) != 2 {
-		t.Fatalf("Expected 2 service entries, got %d", len(server.services))
+	if len(server.services.services) != 2 {
+		t.Fatalf("Expected 2 service entries, got %d", len(server.services.services))
 	}
 
-	svc, ok := server.services["calc"]
+	svc, ok := server.services.services["test"]
 	if !ok {
 		t.Fatalf("Expected service calc to be registered")
 	}
 
-	if len(svc.callbacks) != 5 {
-		t.Errorf("Expected 5 callbacks for service 'calc', got %d", len(svc.callbacks))
-	}
-
-	if len(svc.subscriptions) != 1 {
-		t.Errorf("Expected 1 subscription for service 'calc', got %d", len(svc.subscriptions))
+	wantCallbacks := 7
+	if len(svc.callbacks) != wantCallbacks {
+		t.Errorf("Expected %d callbacks for service 'service', got %d", wantCallbacks, len(svc.callbacks))
 	}
 }
 
-func testServerMethodExecution(t *testing.T, method string) {
-	server := NewServer()
-	service := new(Service)
-
-	if err := server.RegisterName("test", service); err != nil {
-		t.Fatalf("%v", err)
+func TestServer(t *testing.T) {
+	files, err := ioutil.ReadDir("testdata")
+	if err != nil {
+		t.Fatal("where'd my testdata go?")
 	}
+	for _, f := range files {
+		if f.IsDir() || strings.HasPrefix(f.Name(), ".") {
+			continue
+		}
+		path := filepath.Join("testdata", f.Name())
+		name := strings.TrimSuffix(f.Name(), filepath.Ext(f.Name()))
+		t.Run(name, func(t *testing.T) {
+			runTestScript(t, path)
+		})
+	}
+}
 
-	stringArg := "string arg"
-	intArg := 1122
-	argsArg := &Args{"abcde"}
-	params := []interface{}{stringArg, intArg, argsArg}
-
-	request := map[string]interface{}{
-		"id":      12345,
-		"method":  "test_" + method,
-		"version": "2.0",
-		"params":  params,
+func runTestScript(t *testing.T, file string) {
+	server := newTestServer()
+	content, err := ioutil.ReadFile(file)
+	if err != nil {
+		t.Fatal(err)
 	}
 
 	clientConn, serverConn := net.Pipe()
 	defer clientConn.Close()
-
-	go server.ServeCodec(NewJSONCodec(serverConn), OptionMethodInvocation)
-
-	out := json.NewEncoder(clientConn)
-	in := json.NewDecoder(clientConn)
-
-	if err := out.Encode(request); err != nil {
-		t.Fatal(err)
+	go server.ServeCodec(NewJSONCodec(serverConn), OptionMethodInvocation|OptionSubscriptions)
+	readbuf := bufio.NewReader(clientConn)
+	for _, line := range strings.Split(string(content), "\n") {
+		line = strings.TrimSpace(line)
+		switch {
+		case len(line) == 0 || strings.HasPrefix(line, "//"):
+			// skip comments, blank lines
+			continue
+		case strings.HasPrefix(line, "--> "):
+			t.Log(line)
+			// write to connection
+			clientConn.SetWriteDeadline(time.Now().Add(5 * time.Second))
+			if _, err := io.WriteString(clientConn, line[4:]+"\n"); err != nil {
+				t.Fatalf("write error: %v", err)
+			}
+		case strings.HasPrefix(line, "<-- "):
+			t.Log(line)
+			want := line[4:]
+			// read line from connection and compare text
+			clientConn.SetReadDeadline(time.Now().Add(5 * time.Second))
+			sent, err := readbuf.ReadString('\n')
+			if err != nil {
+				t.Fatalf("read error: %v", err)
+			}
+			sent = strings.TrimRight(sent, "\r\n")
+			if sent != want {
+				t.Errorf("wrong line from server\ngot:  %s\nwant: %s", sent, want)
+			}
+		default:
+			panic("invalid line in test script: " + line)
+		}
 	}
+}
 
-	response := jsonSuccessResponse{Result: &Result{}}
-	if err := in.Decode(&response); err != nil {
-		t.Fatal(err)
-	}
+// This test checks that responses are delivered for very short-lived connections that
+// only carry a single request.
+func TestServerShortLivedConn(t *testing.T) {
+	server := newTestServer()
+	defer server.Stop()
 
-	if result, ok := response.Result.(*Result); ok {
-		if result.String != stringArg {
-			t.Errorf("expected %s, got : %s\n", stringArg, result.String)
+	listener, err := net.Listen("tcp", "127.0.0.1:0")
+	if err != nil {
+		t.Fatal("can't listen:", err)
+	}
+	defer listener.Close()
+	go server.ServeListener(listener)
+
+	var (
+		request  = `{"jsonrpc":"2.0","id":1,"method":"rpc_modules"}` + "\n"
+		wantResp = `{"jsonrpc":"2.0","id":1,"result":{"nftest":"1.0","rpc":"1.0","test":"1.0"}}` + "\n"
+		deadline = time.Now().Add(10 * time.Second)
+	)
+	for i := 0; i < 20; i++ {
+		conn, err := net.Dial("tcp", listener.Addr().String())
+		if err != nil {
+			t.Fatal("can't dial:", err)
 		}
-		if result.Int != intArg {
-			t.Errorf("expected %d, got %d\n", intArg, result.Int)
+		defer conn.Close()
+		conn.SetDeadline(deadline)
+		// Write the request, then half-close the connection so the server stops reading.
+		conn.Write([]byte(request))
+		conn.(*net.TCPConn).CloseWrite()
+		// Now try to get the response.
+		buf := make([]byte, 2000)
+		n, err := conn.Read(buf)
+		if err != nil {
+			t.Fatal("read error:", err)
 		}
-		if !reflect.DeepEqual(result.Args, argsArg) {
-			t.Errorf("expected %v, got %v\n", argsArg, result)
+		if !bytes.Equal(buf[:n], []byte(wantResp)) {
+			t.Fatalf("wrong response: %s", buf[:n])
 		}
-	} else {
-		t.Fatalf("invalid response: expected *Result - got: %T", response.Result)
 	}
 }
-
-func TestServerMethodExecution(t *testing.T) {
-	testServerMethodExecution(t, "echo")
-}
-
-func TestServerMethodWithCtx(t *testing.T) {
-	testServerMethodExecution(t, "echoWithCtx")
-}

rpc/service.go

+// Copyright 2015 The go-ethereum Authors
+// This file is part of the go-ethereum library.
+//
+// The go-ethereum library is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Lesser General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// The go-ethereum library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU Lesser General Public License for more details.
+//
+// You should have received a copy of the GNU Lesser General Public License
+// along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
+
+package rpc
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"reflect"
+	"runtime"
+	"strings"
+	"sync"
+	"unicode"
+	"unicode/utf8"
+
+	"github.com/ethereum/go-ethereum/log"
+)
+
+var (
+	contextType      = reflect.TypeOf((*context.Context)(nil)).Elem()
+	errorType        = reflect.TypeOf((*error)(nil)).Elem()
+	subscriptionType = reflect.TypeOf(Subscription{})
+	stringType       = reflect.TypeOf("")
+)
+
+type serviceRegistry struct {
+	mu       sync.Mutex
+	services map[string]service
+}
+
+// service represents a registered object.
+type service struct {
+	name          string               // name for service
+	callbacks     map[string]*callback // registered handlers
+	subscriptions map[string]*callback // available subscriptions/notifications
+}
+
+// callback is a method callback which was registered in the server
+type callback struct {
+	fn          reflect.Value  // the function
+	rcvr        reflect.Value  // receiver object of method, set if fn is method
+	argTypes    []reflect.Type // input argument types
+	hasCtx      bool           // method's first argument is a context (not included in argTypes)
+	errPos      int            // err return idx, of -1 when method cannot return error
+	isSubscribe bool           // true if this is a subscription callback
+}
+
+func (r *serviceRegistry) registerName(name string, rcvr interface{}) error {
+	rcvrVal := reflect.ValueOf(rcvr)
+	if name == "" {
+		return fmt.Errorf("no service name for type %s", rcvrVal.Type().String())
+	}
+	callbacks := suitableCallbacks(rcvrVal)
+	if len(callbacks) == 0 {
+		return fmt.Errorf("service %T doesn't have any suitable methods/subscriptions to expose", rcvr)
+	}
+
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	if r.services == nil {
+		r.services = make(map[string]service)
+	}
+	svc, ok := r.services[name]
+	if !ok {
+		svc = service{
+			name:          name,
+			callbacks:     make(map[string]*callback),
+			subscriptions: make(map[string]*callback),
+		}
+		r.services[name] = svc
+	}
+	for name, cb := range callbacks {
+		if cb.isSubscribe {
+			svc.subscriptions[name] = cb
+		} else {
+			svc.callbacks[name] = cb
+		}
+	}
+	return nil
+}
+
+// callback returns the callback corresponding to the given RPC method name.
+func (r *serviceRegistry) callback(method string) *callback {
+	elem := strings.SplitN(method, serviceMethodSeparator, 2)
+	if len(elem) != 2 {
+		return nil
+	}
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	return r.services[elem[0]].callbacks[elem[1]]
+}
+
+// subscription returns a subscription callback in the given service.
+func (r *serviceRegistry) subscription(service, name string) *callback {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	return r.services[service].subscriptions[name]
+}
+
+// suitableCallbacks iterates over the methods of the given type. It determines if a method
+// satisfies the criteria for a RPC callback or a subscription callback and adds it to the
+// collection of callbacks. See server documentation for a summary of these criteria.
+func suitableCallbacks(receiver reflect.Value) map[string]*callback {
+	typ := receiver.Type()
+	callbacks := make(map[string]*callback)
+	for m := 0; m < typ.NumMethod(); m++ {
+		method := typ.Method(m)
+		if method.PkgPath != "" {
+			continue // method not exported
+		}
+		cb := newCallback(receiver, method.Func)
+		if cb == nil {
+			continue // function invalid
+		}
+		name := formatName(method.Name)
+		callbacks[name] = cb
+	}
+	return callbacks
+}
+
+// newCallback turns fn (a function) into a callback object. It returns nil if the function
+// is unsuitable as an RPC callback.
+func newCallback(receiver, fn reflect.Value) *callback {
+	fntype := fn.Type()
+	c := &callback{fn: fn, rcvr: receiver, errPos: -1, isSubscribe: isPubSub(fntype)}
+	// Determine parameter types. They must all be exported or builtin types.
+	c.makeArgTypes()
+	if !allExportedOrBuiltin(c.argTypes) {
+		return nil
+	}
+	// Verify return types. The function must return at most one error
+	// and/or one other non-error value.
+	outs := make([]reflect.Type, fntype.NumOut())
+	for i := 0; i < fntype.NumOut(); i++ {
+		outs[i] = fntype.Out(i)
+	}
+	if len(outs) > 2 || !allExportedOrBuiltin(outs) {
+		return nil
+	}
+	// If an error is returned, it must be the last returned value.
+	switch {
+	case len(outs) == 1 && isErrorType(outs[0]):
+		c.errPos = 0
+	case len(outs) == 2:
+		if isErrorType(outs[0]) || !isErrorType(outs[1]) {
+			return nil
+		}
+		c.errPos = 1
+	}
+	return c
+}
+
+// makeArgTypes composes the argTypes list.
+func (c *callback) makeArgTypes() {
+	fntype := c.fn.Type()
+	// Skip receiver and context.Context parameter (if present).
+	firstArg := 0
+	if c.rcvr.IsValid() {
+		firstArg++
+	}
+	if fntype.NumIn() > firstArg && fntype.In(firstArg) == contextType {
+		c.hasCtx = true
+		firstArg++
+	}
+	// Add all remaining parameters.
+	c.argTypes = make([]reflect.Type, fntype.NumIn()-firstArg)
+	for i := firstArg; i < fntype.NumIn(); i++ {
+		c.argTypes[i-firstArg] = fntype.In(i)
+	}
+}
+
+// call invokes the callback.
+func (c *callback) call(ctx context.Context, method string, args []reflect.Value) (res interface{}, errRes error) {
+	// Create the argument slice.
+	fullargs := make([]reflect.Value, 0, 2+len(args))
+	if c.rcvr.IsValid() {
+		fullargs = append(fullargs, c.rcvr)
+	}
+	if c.hasCtx {
+		fullargs = append(fullargs, reflect.ValueOf(ctx))
+	}
+	fullargs = append(fullargs, args...)
+
+	// Catch panic while running the callback.
+	defer func() {
+		if err := recover(); err != nil {
+			const size = 64 << 10
+			buf := make([]byte, size)
+			buf = buf[:runtime.Stack(buf, false)]
+			log.Error("RPC method " + method + " crashed: " + fmt.Sprintf("%v\n%s", err, buf))
+			errRes = errors.New("method handler crashed")
+		}
+	}()
+	// Run the callback.
+	results := c.fn.Call(fullargs)
+	if len(results) == 0 {
+		return nil, nil
+	}
+	if c.errPos >= 0 && !results[c.errPos].IsNil() {
+		// Method has returned non-nil error value.
+		err := results[c.errPos].Interface().(error)
+		return reflect.Value{}, err
+	}
+	return results[0].Interface(), nil
+}
+
+// Is this an exported - upper case - name?
+func isExported(name string) bool {
+	rune, _ := utf8.DecodeRuneInString(name)
+	return unicode.IsUpper(rune)
+}
+
+// Are all those types exported or built-in?
+func allExportedOrBuiltin(types []reflect.Type) bool {
+	for _, typ := range types {
+		for typ.Kind() == reflect.Ptr {
+			typ = typ.Elem()
+		}
+		// PkgPath will be non-empty even for an exported type,
+		// so we need to check the type name as well.
+		if !isExported(typ.Name()) && typ.PkgPath() != "" {
+			return false
+		}
+	}
+	return true
+}
+
+// Is t context.Context or *context.Context?
+func isContextType(t reflect.Type) bool {
+	for t.Kind() == reflect.Ptr {
+		t = t.Elem()
+	}
+	return t == contextType
+}
+
+// Does t satisfy the error interface?
+func isErrorType(t reflect.Type) bool {
+	for t.Kind() == reflect.Ptr {
+		t = t.Elem()
+	}
+	return t.Implements(errorType)
+}
+
+// Is t Subscription or *Subscription?
+func isSubscriptionType(t reflect.Type) bool {
+	for t.Kind() == reflect.Ptr {
+		t = t.Elem()
+	}
+	return t == subscriptionType
+}
+
+// isPubSub tests whether the given method has as as first argument a context.Context and
+// returns the pair (Subscription, error).
+func isPubSub(methodType reflect.Type) bool {
+	// numIn(0) is the receiver type
+	if methodType.NumIn() < 2 || methodType.NumOut() != 2 {
+		return false
+	}
+	return isContextType(methodType.In(1)) &&
+		isSubscriptionType(methodType.Out(0)) &&
+		isErrorType(methodType.Out(1))
+}
+
+// formatName converts to first character of name to lowercase.
+func formatName(name string) string {
+	ret := []rune(name)
+	if len(ret) > 0 {
+		ret[0] = unicode.ToLower(ret[0])
+	}
+	return string(ret)
+}

rpc/stdio.go

@@ -26,8 +26,8 @@ import (
 
 // DialStdIO creates a client on stdin/stdout.
 func DialStdIO(ctx context.Context) (*Client, error) {
-	return newClient(ctx, func(_ context.Context) (net.Conn, error) {
-		return stdioConn{}, nil
+	return newClient(ctx, func(_ context.Context) (ServerCodec, error) {
+		return NewJSONCodec(stdioConn{}), nil
 	})
 }
 
@@ -45,20 +45,8 @@ func (io stdioConn) Close() error {
 	return nil
 }
 
-func (io stdioConn) LocalAddr() net.Addr {
-	return &net.UnixAddr{Name: "stdio", Net: "stdio"}
-}
-
-func (io stdioConn) RemoteAddr() net.Addr {
-	return &net.UnixAddr{Name: "stdio", Net: "stdio"}
-}
-
-func (io stdioConn) SetDeadline(t time.Time) error {
-	return &net.OpError{Op: "set", Net: "stdio", Source: nil, Addr: nil, Err: errors.New("deadline not supported")}
-}
-
-func (io stdioConn) SetReadDeadline(t time.Time) error {
-	return &net.OpError{Op: "set", Net: "stdio", Source: nil, Addr: nil, Err: errors.New("deadline not supported")}
+func (io stdioConn) RemoteAddr() string {
+	return "/dev/stdin"
 }
 
 func (io stdioConn) SetWriteDeadline(t time.Time) error {

rpc/testdata/invalid-badid.js

+// This test checks processing of messages with invalid ID.
+
+--> {"id":[],"method":"test_foo"}
+<-- {"jsonrpc":"2.0","id":null,"error":{"code":-32600,"message":"invalid request"}}
+
+--> {"id":{},"method":"test_foo"}
+<-- {"jsonrpc":"2.0","id":null,"error":{"code":-32600,"message":"invalid request"}}

rpc/testdata/invalid-batch.js

+// This test checks the behavior of batches with invalid elements.
+// Empty batches are not allowed. Batches may contain junk.
+
+--> []
+<-- {"jsonrpc":"2.0","id":null,"error":{"code":-32600,"message":"empty batch"}}
+
+--> [1]
+<-- [{"jsonrpc":"2.0","id":null,"error":{"code":-32600,"message":"invalid request"}}]
+
+--> [1,2,3]
+<-- [{"jsonrpc":"2.0","id":null,"error":{"code":-32600,"message":"invalid request"}},{"jsonrpc":"2.0","id":null,"error":{"code":-32600,"message":"invalid request"}},{"jsonrpc":"2.0","id":null,"error":{"code":-32600,"message":"invalid request"}}]
+
+--> [{"jsonrpc":"2.0","id":1,"method":"test_echo","params":["foo",1]},55,{"jsonrpc":"2.0","id":2,"method":"unknown_method"},{"foo":"bar"}]
+<-- [{"jsonrpc":"2.0","id":1,"result":{"String":"foo","Int":1,"Args":null}},{"jsonrpc":"2.0","id":null,"error":{"code":-32600,"message":"invalid request"}},{"jsonrpc":"2.0","id":2,"error":{"code":-32601,"message":"the method unknown_method does not exist/is not available"}},{"jsonrpc":"2.0","id":null,"error":{"code":-32600,"message":"invalid request"}}]

rpc/testdata/invalid-idonly.js

+// This test checks processing of messages that contain just the ID and nothing else.
+
+--> {"id":1}
+<-- {"jsonrpc":"2.0","id":1,"error":{"code":-32600,"message":"invalid request"}}
+
+--> {"jsonrpc":"2.0","id":1}
+<-- {"jsonrpc":"2.0","id":1,"error":{"code":-32600,"message":"invalid request"}}

rpc/testdata/invalid-nonobj.js

+// This test checks behavior for invalid requests.
+
+--> 1
+<-- {"jsonrpc":"2.0","id":null,"error":{"code":-32600,"message":"invalid request"}}

rpc/testdata/invalid-syntax.json

+// This test checks that an error is written for invalid JSON requests.
+
+--> 'f
+<-- {"jsonrpc":"2.0","id":null,"error":{"code":-32700,"message":"invalid character '\\'' looking for beginning of value"}}
+

rpc/testdata/reqresp-batch.js

+// There is no response for all-notification batches.
+
+--> [{"jsonrpc":"2.0","method":"test_echo","params":["x",99]}]
+
+// This test checks regular batch calls.
+
+--> [{"jsonrpc":"2.0","id":2,"method":"test_echo","params":[]}, {"jsonrpc":"2.0","id": 3,"method":"test_echo","params":["x",3]}]
+<-- [{"jsonrpc":"2.0","id":2,"error":{"code":-32602,"message":"missing value for required argument 0"}},{"jsonrpc":"2.0","id":3,"result":{"String":"x","Int":3,"Args":null}}]

rpc/testdata/reqresp-echo.js

+// This test calls the test_echo method.
+
+--> {"jsonrpc": "2.0", "id": 2, "method": "test_echo", "params": []}
+<-- {"jsonrpc":"2.0","id":2,"error":{"code":-32602,"message":"missing value for required argument 0"}}
+
+--> {"jsonrpc": "2.0", "id": 2, "method": "test_echo", "params": ["x"]}
+<-- {"jsonrpc":"2.0","id":2,"error":{"code":-32602,"message":"missing value for required argument 1"}}
+
+--> {"jsonrpc": "2.0", "id": 2, "method": "test_echo", "params": ["x", 3]}
+<-- {"jsonrpc":"2.0","id":2,"result":{"String":"x","Int":3,"Args":null}}
+
+--> {"jsonrpc": "2.0", "id": 2, "method": "test_echo", "params": ["x", 3, {"S": "foo"}]}
+<-- {"jsonrpc":"2.0","id":2,"result":{"String":"x","Int":3,"Args":{"S":"foo"}}}
+
+--> {"jsonrpc": "2.0", "id": 2, "method": "test_echoWithCtx", "params": ["x", 3, {"S": "foo"}]}
+<-- {"jsonrpc":"2.0","id":2,"result":{"String":"x","Int":3,"Args":{"S":"foo"}}}

rpc/testdata/reqresp-namedparam.js

+// This test checks that an error response is sent for calls
+// with named parameters. 
+
+--> {"jsonrpc":"2.0","method":"test_echo","params":{"int":23},"id":3}
+<-- {"jsonrpc":"2.0","id":3,"error":{"code":-32602,"message":"non-array args"}}

rpc/testdata/reqresp-noargsrets.js

+// This test calls the test_noArgsRets method.
+
+--> {"jsonrpc": "2.0", "id": "foo", "method": "test_noArgsRets", "params": []}
+<-- {"jsonrpc":"2.0","id":"foo","result":null}

rpc/testdata/reqresp-nomethod.js

+// This test calls a method that doesn't exist.
+
+--> {"jsonrpc": "2.0", "id": 2, "method": "invalid_method", "params": [2, 3]}
+<-- {"jsonrpc":"2.0","id":2,"error":{"code":-32601,"message":"the method invalid_method does not exist/is not available"}}

rpc/testdata/reqresp-noparam.js

+// This test checks that calls with no parameters work.
+
+--> {"jsonrpc":"2.0","method":"test_noArgsRets","id":3}
+<-- {"jsonrpc":"2.0","id":3,"result":null}

rpc/testdata/reqresp-paramsnull.js

+// This test checks that calls with "params":null work.
+
+--> {"jsonrpc":"2.0","method":"test_noArgsRets","params":null,"id":3}
+<-- {"jsonrpc":"2.0","id":3,"result":null}

rpc/testdata/revcall.js

+// This test checks reverse calls.
+
+--> {"jsonrpc":"2.0","id":2,"method":"test_callMeBack","params":["foo",[1]]}
+<-- {"jsonrpc":"2.0","id":1,"method":"foo","params":[1]}
+--> {"jsonrpc":"2.0","id":1,"result":"my result"}
+<-- {"jsonrpc":"2.0","id":2,"result":"my result"}

rpc/testdata/revcall2.js

+// This test checks reverse calls.
+
+--> {"jsonrpc":"2.0","id":2,"method":"test_callMeBackLater","params":["foo",[1]]}
+<-- {"jsonrpc":"2.0","id":2,"result":null}
+<-- {"jsonrpc":"2.0","id":1,"method":"foo","params":[1]}
+--> {"jsonrpc":"2.0","id":1,"result":"my result"}
+

rpc/testdata/subscription.js

+// This test checks basic subscription support.
+
+--> {"jsonrpc":"2.0","id":1,"method":"nftest_subscribe","params":["someSubscription",5,1]}
+<-- {"jsonrpc":"2.0","id":1,"result":"0x1"}
+<-- {"jsonrpc":"2.0","method":"nftest_subscription","params":{"subscription":"0x1","result":1}}
+<-- {"jsonrpc":"2.0","method":"nftest_subscription","params":{"subscription":"0x1","result":2}}
+<-- {"jsonrpc":"2.0","method":"nftest_subscription","params":{"subscription":"0x1","result":3}}
+<-- {"jsonrpc":"2.0","method":"nftest_subscription","params":{"subscription":"0x1","result":4}}
+<-- {"jsonrpc":"2.0","method":"nftest_subscription","params":{"subscription":"0x1","result":5}}
+
+--> {"jsonrpc":"2.0","id":2,"method":"nftest_echo","params":[11]}
+<-- {"jsonrpc":"2.0","id":2,"result":11}

rpc/testservice_test.go

+// Copyright 2018 The go-ethereum Authors
+// This file is part of the go-ethereum library.
+//
+// The go-ethereum library is free software: you can redistribute it and/or modify
+// it under the terms of the GNU Lesser General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// The go-ethereum library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU Lesser General Public License for more details.
+//
+// You should have received a copy of the GNU Lesser General Public License
+// along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
+
+package rpc
+
+import (
+	"context"
+	"encoding/binary"
+	"errors"
+	"sync"
+	"time"
+)
+
+func newTestServer() *Server {
+	server := NewServer()
+	server.idgen = sequentialIDGenerator()
+	if err := server.RegisterName("test", new(testService)); err != nil {
+		panic(err)
+	}
+	if err := server.RegisterName("nftest", new(notificationTestService)); err != nil {
+		panic(err)
+	}
+	return server
+}
+
+func sequentialIDGenerator() func() ID {
+	var (
+		mu      sync.Mutex
+		counter uint64
+	)
+	return func() ID {
+		mu.Lock()
+		defer mu.Unlock()
+		counter++
+		id := make([]byte, 8)
+		binary.BigEndian.PutUint64(id, counter)
+		return encodeID(id)
+	}
+}
+
+type testService struct{}
+
+type Args struct {
+	S string
+}
+
+type Result struct {
+	String string
+	Int    int
+	Args   *Args
+}
+
+func (s *testService) NoArgsRets() {}
+
+func (s *testService) Echo(str string, i int, args *Args) Result {
+	return Result{str, i, args}
+}
+
+func (s *testService) EchoWithCtx(ctx context.Context, str string, i int, args *Args) Result {
+	return Result{str, i, args}
+}
+
+func (s *testService) Sleep(ctx context.Context, duration time.Duration) {
+	time.Sleep(duration)
+}
+
+func (s *testService) Rets() (string, error) {
+	return "", nil
+}
+
+func (s *testService) InvalidRets1() (error, string) {
+	return nil, ""
+}
+
+func (s *testService) InvalidRets2() (string, string) {
+	return "", ""
+}
+
+func (s *testService) InvalidRets3() (string, string, error) {
+	return "", "", nil
+}
+
+func (s *testService) CallMeBack(ctx context.Context, method string, args []interface{}) (interface{}, error) {
+	c, ok := ClientFromContext(ctx)
+	if !ok {
+		return nil, errors.New("no client")
+	}
+	var result interface{}
+	err := c.Call(&result, method, args...)
+	return result, err
+}
+
+func (s *testService) CallMeBackLater(ctx context.Context, method string, args []interface{}) error {
+	c, ok := ClientFromContext(ctx)
+	if !ok {
+		return errors.New("no client")
+	}
+	go func() {
+		<-ctx.Done()
+		var result interface{}
+		c.Call(&result, method, args...)
+	}()
+	return nil
+}
+
+func (s *testService) Subscription(ctx context.Context) (*Subscription, error) {
+	return nil, nil
+}
+
+type notificationTestService struct {
+	unsubscribed            chan string
+	gotHangSubscriptionReq  chan struct{}
+	unblockHangSubscription chan struct{}
+}
+
+func (s *notificationTestService) Echo(i int) int {
+	return i
+}
+
+func (s *notificationTestService) Unsubscribe(subid string) {
+	if s.unsubscribed != nil {
+		s.unsubscribed <- subid
+	}
+}
+
+func (s *notificationTestService) SomeSubscription(ctx context.Context, n, val int) (*Subscription, error) {
+	notifier, supported := NotifierFromContext(ctx)
+	if !supported {
+		return nil, ErrNotificationsUnsupported
+	}
+
+	// By explicitly creating an subscription we make sure that the subscription id is send
+	// back to the client before the first subscription.Notify is called. Otherwise the
+	// events might be send before the response for the *_subscribe method.
+	subscription := notifier.CreateSubscription()
+	go func() {
+		for i := 0; i < n; i++ {
+			if err := notifier.Notify(subscription.ID, val+i); err != nil {
+				return
+			}
+		}
+		select {
+		case <-notifier.Closed():
+		case <-subscription.Err():
+		}
+		if s.unsubscribed != nil {
+			s.unsubscribed <- string(subscription.ID)
+		}
+	}()
+	return subscription, nil
+}
+
+// HangSubscription blocks on s.unblockHangSubscription before sending anything.
+func (s *notificationTestService) HangSubscription(ctx context.Context, val int) (*Subscription, error) {
+	notifier, supported := NotifierFromContext(ctx)
+	if !supported {
+		return nil, ErrNotificationsUnsupported
+	}
+	s.gotHangSubscriptionReq <- struct{}{}
+	<-s.unblockHangSubscription
+	subscription := notifier.CreateSubscription()
+
+	go func() {
+		notifier.Notify(subscription.ID, val)
+	}()
+	return subscription, nil
+}

rpc/types.go

@@ -17,13 +17,11 @@
 package rpc
 
 import (
+	"context"
 	"fmt"
 	"math"
-	"reflect"
 	"strings"
-	"sync"
 
-	mapset "github.com/deckarep/golang-set"
 	"github.com/ethereum/go-ethereum/common/hexutil"
 )
 
@@ -35,57 +33,6 @@ type API struct {
 	Public    bool        // indication if the methods must be considered safe for public use
 }
 
-// callback is a method callback which was registered in the server
-type callback struct {
-	rcvr        reflect.Value  // receiver of method
-	method      reflect.Method // callback
-	argTypes    []reflect.Type // input argument types
-	hasCtx      bool           // method's first argument is a context (not included in argTypes)
-	errPos      int            // err return idx, of -1 when method cannot return error
-	isSubscribe bool           // indication if the callback is a subscription
-}
-
-// service represents a registered object
-type service struct {
-	name          string        // name for service
-	typ           reflect.Type  // receiver type
-	callbacks     callbacks     // registered handlers
-	subscriptions subscriptions // available subscriptions/notifications
-}
-
-// serverRequest is an incoming request
-type serverRequest struct {
-	id            interface{}
-	svcname       string
-	callb         *callback
-	args          []reflect.Value
-	isUnsubscribe bool
-	err           Error
-}
-
-type serviceRegistry map[string]*service // collection of services
-type callbacks map[string]*callback      // collection of RPC callbacks
-type subscriptions map[string]*callback  // collection of subscription callbacks
-
-// Server represents a RPC server
-type Server struct {
-	services serviceRegistry
-
-	run      int32
-	codecsMu sync.Mutex
-	codecs   mapset.Set
-}
-
-// rpcRequest represents a raw incoming RPC request
-type rpcRequest struct {
-	service  string
-	method   string
-	id       interface{}
-	isPubSub bool
-	params   interface{}
-	err      Error // invalid batch element
-}
-
 // Error wraps RPC errors, which contain an error code in addition to the message.
 type Error interface {
 	Error() string  // returns the message
@@ -96,24 +43,19 @@ type Error interface {
 // a RPC session. Implementations must be go-routine safe since the codec can be called in
 // multiple go-routines concurrently.
 type ServerCodec interface {
-	// Read next request
-	ReadRequestHeaders() ([]rpcRequest, bool, Error)
-	// Parse request argument to the given types
-	ParseRequestArguments(argTypes []reflect.Type, params interface{}) ([]reflect.Value, Error)
-	// Assemble success response, expects response id and payload
-	CreateResponse(id interface{}, reply interface{}) interface{}
-	// Assemble error response, expects response id and error
-	CreateErrorResponse(id interface{}, err Error) interface{}
-	// Assemble error response with extra information about the error through info
-	CreateErrorResponseWithInfo(id interface{}, err Error, info interface{}) interface{}
-	// Create notification response
-	CreateNotification(id, namespace string, event interface{}) interface{}
-	// Write msg to client.
-	Write(msg interface{}) error
-	// Close underlying data stream
+	Read() (msgs []*jsonrpcMessage, isBatch bool, err error)
 	Close()
-	// Closed when underlying connection is closed
+	jsonWriter
+}
+
+// jsonWriter can write JSON messages to its underlying connection.
+// Implementations must be safe for concurrent use.
+type jsonWriter interface {
+	Write(context.Context, interface{}) error
+	// Closed returns a channel which is closed when the connection is closed.
 	Closed() <-chan interface{}
+	// RemoteAddr returns the peer address of the connection.
+	RemoteAddr() string
 }
 
 type BlockNumber int64

rpc/utils.go

-// Copyright 2015 The go-ethereum Authors
-// This file is part of the go-ethereum library.
-//
-// The go-ethereum library is free software: you can redistribute it and/or modify
-// it under the terms of the GNU Lesser General Public License as published by
-// the Free Software Foundation, either version 3 of the License, or
-// (at your option) any later version.
-//
-// The go-ethereum library is distributed in the hope that it will be useful,
-// but WITHOUT ANY WARRANTY; without even the implied warranty of
-// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-// GNU Lesser General Public License for more details.
-//
-// You should have received a copy of the GNU Lesser General Public License
-// along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
-
-package rpc
-
-import (
-	"bufio"
-	"context"
-	crand "crypto/rand"
-	"encoding/binary"
-	"encoding/hex"
-	"math/rand"
-	"reflect"
-	"strings"
-	"sync"
-	"time"
-	"unicode"
-	"unicode/utf8"
-)
-
-var (
-	subscriptionIDGenMu sync.Mutex
-	subscriptionIDGen   = idGenerator()
-)
-
-// Is this an exported - upper case - name?
-func isExported(name string) bool {
-	rune, _ := utf8.DecodeRuneInString(name)
-	return unicode.IsUpper(rune)
-}
-
-// Is this type exported or a builtin?
-func isExportedOrBuiltinType(t reflect.Type) bool {
-	for t.Kind() == reflect.Ptr {
-		t = t.Elem()
-	}
-	// PkgPath will be non-empty even for an exported type,
-	// so we need to check the type name as well.
-	return isExported(t.Name()) || t.PkgPath() == ""
-}
-
-var contextType = reflect.TypeOf((*context.Context)(nil)).Elem()
-
-// isContextType returns an indication if the given t is of context.Context or *context.Context type
-func isContextType(t reflect.Type) bool {
-	for t.Kind() == reflect.Ptr {
-		t = t.Elem()
-	}
-	return t == contextType
-}
-
-var errorType = reflect.TypeOf((*error)(nil)).Elem()
-
-// Implements this type the error interface
-func isErrorType(t reflect.Type) bool {
-	for t.Kind() == reflect.Ptr {
-		t = t.Elem()
-	}
-	return t.Implements(errorType)
-}
-
-var subscriptionType = reflect.TypeOf((*Subscription)(nil)).Elem()
-
-// isSubscriptionType returns an indication if the given t is of Subscription or *Subscription type
-func isSubscriptionType(t reflect.Type) bool {
-	for t.Kind() == reflect.Ptr {
-		t = t.Elem()
-	}
-	return t == subscriptionType
-}
-
-// isPubSub tests whether the given method has as as first argument a context.Context
-// and returns the pair (Subscription, error)
-func isPubSub(methodType reflect.Type) bool {
-	// numIn(0) is the receiver type
-	if methodType.NumIn() < 2 || methodType.NumOut() != 2 {
-		return false
-	}
-
-	return isContextType(methodType.In(1)) &&
-		isSubscriptionType(methodType.Out(0)) &&
-		isErrorType(methodType.Out(1))
-}
-
-// formatName will convert to first character to lower case
-func formatName(name string) string {
-	ret := []rune(name)
-	if len(ret) > 0 {
-		ret[0] = unicode.ToLower(ret[0])
-	}
-	return string(ret)
-}
-
-// suitableCallbacks iterates over the methods of the given type. It will determine if a method satisfies the criteria
-// for a RPC callback or a subscription callback and adds it to the collection of callbacks or subscriptions. See server
-// documentation for a summary of these criteria.
-func suitableCallbacks(rcvr reflect.Value, typ reflect.Type) (callbacks, subscriptions) {
-	callbacks := make(callbacks)
-	subscriptions := make(subscriptions)
-
-METHODS:
-	for m := 0; m < typ.NumMethod(); m++ {
-		method := typ.Method(m)
-		mtype := method.Type
-		mname := formatName(method.Name)
-		if method.PkgPath != "" { // method must be exported
-			continue
-		}
-
-		var h callback
-		h.isSubscribe = isPubSub(mtype)
-		h.rcvr = rcvr
-		h.method = method
-		h.errPos = -1
-
-		firstArg := 1
-		numIn := mtype.NumIn()
-		if numIn >= 2 && mtype.In(1) == contextType {
-			h.hasCtx = true
-			firstArg = 2
-		}
-
-		if h.isSubscribe {
-			h.argTypes = make([]reflect.Type, numIn-firstArg) // skip rcvr type
-			for i := firstArg; i < numIn; i++ {
-				argType := mtype.In(i)
-				if isExportedOrBuiltinType(argType) {
-					h.argTypes[i-firstArg] = argType
-				} else {
-					continue METHODS
-				}
-			}
-
-			subscriptions[mname] = &h
-			continue METHODS
-		}
-
-		// determine method arguments, ignore first arg since it's the receiver type
-		// Arguments must be exported or builtin types
-		h.argTypes = make([]reflect.Type, numIn-firstArg)
-		for i := firstArg; i < numIn; i++ {
-			argType := mtype.In(i)
-			if !isExportedOrBuiltinType(argType) {
-				continue METHODS
-			}
-			h.argTypes[i-firstArg] = argType
-		}
-
-		// check that all returned values are exported or builtin types
-		for i := 0; i < mtype.NumOut(); i++ {
-			if !isExportedOrBuiltinType(mtype.Out(i)) {
-				continue METHODS
-			}
-		}
-
-		// when a method returns an error it must be the last returned value
-		h.errPos = -1
-		for i := 0; i < mtype.NumOut(); i++ {
-			if isErrorType(mtype.Out(i)) {
-				h.errPos = i
-				break
-			}
-		}
-
-		if h.errPos >= 0 && h.errPos != mtype.NumOut()-1 {
-			continue METHODS
-		}
-
-		switch mtype.NumOut() {
-		case 0, 1, 2:
-			if mtype.NumOut() == 2 && h.errPos == -1 { // method must one return value and 1 error
-				continue METHODS
-			}
-			callbacks[mname] = &h
-		}
-	}
-
-	return callbacks, subscriptions
-}
-
-// idGenerator helper utility that generates a (pseudo) random sequence of
-// bytes that are used to generate identifiers.
-func idGenerator() *rand.Rand {
-	if seed, err := binary.ReadVarint(bufio.NewReader(crand.Reader)); err == nil {
-		return rand.New(rand.NewSource(seed))
-	}
-	return rand.New(rand.NewSource(int64(time.Now().Nanosecond())))
-}
-
-// NewID generates a identifier that can be used as an identifier in the RPC interface.
-// e.g. filter and subscription identifier.
-func NewID() ID {
-	subscriptionIDGenMu.Lock()
-	defer subscriptionIDGenMu.Unlock()
-
-	id := make([]byte, 16)
-	for i := 0; i < len(id); i += 7 {
-		val := subscriptionIDGen.Int63()
-		for j := 0; i+j < len(id) && j < 7; j++ {
-			id[i+j] = byte(val)
-			val >>= 8
-		}
-	}
-
-	rpcId := hex.EncodeToString(id)
-	// rpc ID's are RPC quantities, no leading zero's and 0 is 0x0
-	rpcId = strings.TrimLeft(rpcId, "0")
-	if rpcId == "" {
-		rpcId = "0"
-	}
-
-	return ID("0x" + rpcId)
-}