Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open API ignores pg type comment #1377

Open
fstn opened this issue Aug 13, 2019 · 6 comments
Open

Open API ignores pg type comment #1377

fstn opened this issue Aug 13, 2019 · 6 comments
Labels

Comments

@fstn
Copy link

@fstn fstn commented Aug 13, 2019

I declared my procedure as:

CREATE TYPE user_acquisition_search_type AS (
    min_date_range text,
    max_date_range text,
    event text,
    pixel_key text);

COMMENT ON type user_acquisition_search_type IS
    'min_date_range text,
    max_date_range text,
    event text,
    pixel_key text';

CREATE OR REPLACE FUNCTION user_acquisition(search user_acquisition_search_type)
    RETURNS JSONB AS
$$

When I check the generated open api, I get:

 "/rpc/user_acquisition": {
      "post": {
        "tags": [
          "(rpc) user_acquisition"
        ],
        "summary": " TOTAL USER ACQUISITION ",
        "description": "   ",
        "produces": [
          "application/json",
          "application/vnd.pgrst.object+json"
        ],
        "parameters": [
          {
            "required": true,
            "schema": {
              "required": [
                "search"
              ],
              "type": "object",
              "description": "   ",
              "properties": {
                "search": {
                  "format": "total_user_acquisition_search_type",
                  "type": "string"
                }
              }
            },
            "in": "body",
            "name": "args"
          },....

I should have my real search user_acquisition_search_type and the comment I applied on it into the openapi json file instead of nothing.

@nerfpops
Copy link

@nerfpops nerfpops commented Aug 26, 2019

The reason seems to be

toSwaggerType :: Text -> SwaggerType t
toSwaggerType "character varying" = SwaggerString
toSwaggerType "character" = SwaggerString
toSwaggerType "text" = SwaggerString
toSwaggerType "boolean" = SwaggerBoolean
toSwaggerType "smallint" = SwaggerInteger
toSwaggerType "integer" = SwaggerInteger
toSwaggerType "bigint" = SwaggerInteger
toSwaggerType "numeric" = SwaggerNumber
toSwaggerType "real" = SwaggerNumber
toSwaggerType "double precision" = SwaggerNumber
toSwaggerType _ = SwaggerString

having a predefined list of supported types, where everything else is treated as a string. And also

makeProcProperty :: PgArg -> (Text, Referenced Schema)
makeProcProperty (PgArg n t _) = (n, Inline s)
where
s = (mempty :: Schema)
& type_ .~ toSwaggerType t
& format ?~ t

which doesn't support objects.

If the function is immutable, it would be accessible via a GET request, which would probably need to be modeled somehow in Swagger to match query strings like search.event=a&search.pixel_key=b (with each property marked as the appropriate type), and a model supporting POST requests, where the parameters probably would be something like { "search": { "event": "a", "pixel_key": "b", ... } }. Just guessing...

I'd assume what's in your comment should simply be infered from the type. No need for a comment with type definitions.

Regarding your title: I initially thought you meant a Postgres PROCEDURE introduced in v11. Took me a while to figure out you meant your FUNCTION argument type total_user_acquisition_search_type didn't get added correctly.

I have never written a line of Haskell in my life, so I probably can't help you though.

@steve-chavez
Copy link
Member

@steve-chavez steve-chavez commented Aug 27, 2019

Thanks for the help here @nerfpops. You're right in that we default to a string for unknown pg types.

@fstn I see you want to document the function arguments. As a workaround, perhaps you could COMMENT on the function instead?

@fstn
Copy link
Author

@fstn fstn commented Aug 27, 2019

@steve-chavez steve-chavez changed the title Open API ignore procedure type comment Open API ignores pg type comment Aug 27, 2019
@nerfpops
Copy link

@nerfpops nerfpops commented Aug 27, 2019

Hi @steve-chavez, thank you so much for all the work you put into this project. I see that you renamed the issue, but shouldn't postgrest ideally spit out a schema matching the function signature, even with composite types?

@nerfpops
Copy link

@nerfpops nerfpops commented Aug 27, 2019

I thought this would be a simple starter issue for learning Haskell, but it seems to not be that easy after all.

I'm trying to figure out how to add type comments in openapi, but to do so, I'm imagining something like the following will have to be done...

Currently, these values are preloaded:

getDbStructure :: Schema -> PgVersion -> HT.Transaction DbStructure
getDbStructure schema pgVer = do
HT.sql "set local schema ''" -- for getting the fully qualified name(schema.name) of every db object
tabs <- HT.statement () allTables
cols <- HT.statement schema $ allColumns tabs
syns <- HT.statement schema $ allSynonyms cols pgVer
childRels <- HT.statement () $ allChildRelations tabs cols
keys <- HT.statement () $ allPrimaryKeys tabs
procs <- HT.statement schema allProcs

wouldn't it make sense to preload all the types as well, with their comments?

PgType would need to contain all the required information for a type, and no longer just a QualifiedIdentifier

data PgType = Scalar QualifiedIdentifier | Composite QualifiedIdentifier deriving (Eq, Show, Ord)

My noob skills in Haskell is going to show it's ugly face here, but to enable us to print comments, and also the type information, PgType would need to look something like this:

data PgTypeDescription = PgTypeDescription {
    ptdSchema :: Text
  , ptdName :: Text
  , ptdDescription :: Maybe Text
} deriving (Eq, Show, Ord)

-- Base type name in Postgres (text/jsonb/uuid/...)
data PgBaseType = Text deriving (Eq, Show, Ord)

data PgType
  = Scalar PgTypeDescription PgBaseType
  | Composite PgTypeDescription [PgType]
  deriving (Eq, Show, Ord)

With [PgType] added to Composite because you can't create composite types without columns.

When preloading types, it must be each and every type in the database, since you can reference any type outside the public schema.

Would this be a viable approach?

Edit: On second thought, I'll probably get in trouble with circular references, if I add [PgType] to Composite. I should stick with

data PgTypeDescription = PgTypeDescription {
    ptdSchema :: Text
  , ptdName :: Text
  , ptdDescription :: Maybe Text
} deriving (Eq, Show, Ord)

data PgType
  = Scalar PgTypeDescription
  | Composite PgTypeDescription
  deriving (Eq, Show, Ord)
@steve-chavez
Copy link
Member

@steve-chavez steve-chavez commented Sep 2, 2019

@nerfpops Seems like a solid approach.

You'd need to query pg_type first and then convert to Haskell types. A type can be created on a particular schema. At first I was thinking we should only get the exposed db-schema types. But getting all of the db types seems fine.

As a first step it'd be better to focus on getting only the types comments. A nice mapping between the pg type and haskell type could get cumbersome as you mentioned. This could be refined later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.