Suggestion: cleaner definition types

[andykais]

Hi! Super excited about this library, gRPC was too unapproachable no its own but this library has a much smaller footprint which is awesome. One thing that stuck out though was your service definitions. They look a tad awkward. Messing around, I figured out how to convert handlers into definitions using typescript object traversal. I was wondering if this could be useful in some capacity.

import { NodeHttpTransport } from '@improbable-eng/grpc-web-node-http-transport'
import { ModuleRpcCommon } from 'rpc_ts/lib/common'
import { ModuleRpcServer } from 'rpc_ts/lib/server'
import { ModuleRpcProtocolServer } from 'rpc_ts/lib/protocol/server'
import { ModuleRpcProtocolClient } from 'rpc_ts/lib/protocol/client'

type ArgumentTypes<F extends Function> = F extends (...args: infer A) => any ? A : never
type ThenArg<T> = T extends Promise<infer U> ? U : T

type ServiceHandlers = { [fnName: string]: (...args: any[]) => any }
type ConvertHandlersToDefinitions<I extends ServiceHandlers> = {
  [K in keyof I]: {
    request: ArgumentTypes<I[K]>[0]
    response: ThenArg<ReturnType<I[K]>>
  }
}
const createServiceDefinitions = <T extends ServiceHandlers>(
  serviceHandlers: T
): ConvertHandlersToDefinitions<T> => {
  const definitions: any = Object.entries(serviceHandlers).reduce((acc, [key, fn]) => {
    acc[key] = {
      request: {},
      response: {}
    }
    return acc
  }, {})
  return definitions
}

// Definition of the RPC service
const helloServiceHandlers = {
  getHello: async (request: { language: string }) => {
    const { language } = request
    if (language === 'Spanish') return { text: 'Hola' }
    throw new ModuleRpcServer.ServerRpcError(
      ModuleRpcCommon.RpcErrorType.notFound,
      `language '${language}' not found`
    )
  }
}

// Implementation of an RPC server
import * as express from 'express'
import * as http from 'http'
const app = express()

export const helloServiceDefinition = createServiceDefinitions(helloServiceHandlers)
app.use(ModuleRpcProtocolServer.registerRpcRoutes(helloServiceDefinition, helloServiceHandlers))

const server = http.createServer(app).listen(4000, () => {
  console.log('server started')
})

Personally I like the idea of defining my protocols once instead of a type and then a handler. However, if you think defnitions should be the source of truth (like gRPC) then you could define a type only in typescript and pass that around instead of this definitions object that also holds the definition types

[andykais]

finished a more complete example of the latter (interface is the single source of truth, clients and handlers are based off of them). It includes both a handler and a client. It has to use the handlers to create the clients though, which is still a tad awkward

typescript playground

[andykais]

heres a slightly stranger option that would reduce the boilerplate but would require the use of a Proxy on the client.

typescript playground

[lingz]

@andykais thanks for your detailed typescript playground examples!

I can however, clearly see that it is possible to define only a handler once, and have the service types be inferred. However, I believe the original reason the handlers and service definitions are separate is:

We use the RPC client only on the React Web UI, whereas the Server side handlers implement the handlers using server side specific packages (like a DB driver). Both sides import the service definition, but neither side imports each other.

Now, it's possible with good tree shaking, or stubbing out webpack builds, or a typescript -> webpack build, to have the Web UI import the service definition types only without implementation. However, this could present more challenges than it's worth, and is not trivial to do depending on how you've set up the build.

Therefore, the simple solution is to have the service definition to have decoupling of the interfaces. In a microservices multi-repo based approach, the definitions could even live in their own repository, with the RPC client not even able to reach the handler code at all.

Let me know if your use case falls outside this.

[andykais]

Ah fair enough, I had not considered the annoyances of tree shaking. The third example (using a Proxy) could still make it more clear by defining a type only interface, not a type-object combo, but I can see how I might be over-engineering a bit just to avoid that. In any case, I think I need to integrate my solution into my web framework pretty tightly (I am using svelte/sapper, which has a different fetch implementation for SSR) so I will likely have to write a custom RPC lib. I did succeed in creating a simple rpc client & handlers in under 200 LOC without the use of gRPC though that serves my purposes. I can post it here if that is of interest.

[lishine]

@andykais I am interested in your RPC implementation. May you put it in a gist?

[lishine]

@lingz
what is actually the reason to use grpc while sending json? What are the grpc+json advantages?

How can I gain understanding of the lib usage and implementation? I mean, should I look into grpc-web documentation?

What actually your lib add or customises beyond grpc-web?

First thing I need to know is how to customise headers and send cookies.

Where can I read about how this grpc+json implementation and how it's way of operation differs from regular POST?

• grpc-web documentation seems scars about json. It is mostly about protobuf

Thank you!

[lishine]

How do you do dynamic validation?

[lishine]

Regular POST can be sent currently over http2, while grpc-web cannot, right?

[lishine]

Why no need for proxy?

[andykais]

@lishine I actually have a full working implementation of what I described, but unfortunately I am waiting for it to be cleared with my company before I opens source it. What I can share right now, is that this is fairly easy to roll yourself w/ proxies. As for dynamic validation, if you want to stick with pure typescript types, theres a fantastic library for creating runtime validators from typescript types here https://www.npmjs.com/package/typescript-is.

Finally, I will just leave some general advice about RPCs. They work fantastic for client/server implementations where the surface api is relatively small, and where the client and server are both written by you. It keeps your code DRY. Adding dynamic validation really implies that the api is open to the public and you want to be ready for any input under the sun. This is where RPCs start to get unwieldy. GraphQL & REST apis are more suited as client facing apis since they have well known verbage and allow more flexible usage. In those cases, its actually beneficial to not share an implementation on the client and server.

[lishine]

@andykais thanks for your help
You meant W/O proxy?
What about grpc. It is meant to share proto types between server and client. I saw YouTube of Badoo, they big very big and they use grpc with some adaptation and including json grpc.
What is the difference between RPC with typescript types sharing and Graphql/Rest? Maybe you mean if some non typescript client want to use it?

[andykais]

I dont want to hijack this thread too much since we're getting off topic but I will try to answer concisely:

You meant W/O proxy?

the solution I built uses proxies (it has compile time safety, but no runtime safety)

What about grpc

there are definitely large apps built with grpc, if you wish to go this route that is certainly an option, but I can argue that the implementation will be more complex than a simple typescript rpc, specifically because it has to support so many different usage patterns

What is the difference between RPC with typescript types sharing and Graphql/Rest? Maybe you mean if some non typescript client want to use it?

It is like you said, a rpc implementation using only typescript can rely on compile time checks for safety, but if you do not know if your clients will all use typescript, then you will want some kind of runtime safety, hence grpc, graphql, rest, etc