Publish Message to Pub/Sub Topic With Schema

Hi all i had a topic with schema that used for bigquery subscription
my topic schema is like this

resource "google_pubsub_schema" "my_schema_name" {
  name       = "my_schema_name"
  type       = "PROTOCOL_BUFFER"
  definition = <<EOF
syntax = "proto2";
message ProtocolBuffer {
  required string id = 1;
  required int64 profile_id = 2;
  optional string address = 3;
  required string date = 5;
}
EOF
}

and my BQ table

id | String | required
profile_id | Integer | required
address | String | nullable
date | Date | required

i have been successfully publish message to the topic and the subscription automatically insert it to the table using google pub/sub library
but i have a case where i need to publish it via https

curl --location --request POST 'https://pubsub.googleapis.com/v1/projects/my-project/topics/my-topic:publish' \
--header 'Authorization: Bearer myAccessToken' \
--header 'Content-Type: application/json' \
--data-raw '{
    "messages": [
        {
            "attributes": {
                "id": "ID123",
                "profile_id": 123,
                "address": "somewhere in the world",
                "date": "2023-07-20",
            }
        }
    ]
}'

with curl above i got this error

{
    "error": {
        "code": 400,
        "message": "Invalid JSON payload received. Unknown name \"intValue\" at 'messages[0].attributes[2].value': Cannot find field.",
        "status": "INVALID_ARGUMENT",
        "details": [
            {
                "@type": "type.googleapis.com/google.rpc.BadRequest",
                "fieldViolations": [
                    {
                        "field": "messages[0].attributes[2].value",
                        "description": "Invalid JSON payload received. Unknown name \"intValue\" at 'messages[0].attributes[2].value': Cannot find field."
                    }
                ]
            }
        ]
    }
}

if i quote the profile id

curl --location --request POST 'https://pubsub.googleapis.com/v1/projects/my-project/topics/my-topic:publish' \
--header 'Authorization: Bearer myAccessToken' \
--header 'Content-Type: application/json' \
--data-raw '{
    "messages": [
        {
            "attributes": {
                "id": "ID123",
                "profile_id": "123",
                "address": "somewhere in the world",
                "date": "2023-07-20",
            }
        }
    ]
}'

i got this error instead

{
    "error": {
        "code": 400,
        "message": "Invalid data in message: Message failed schema validation.",
        "status": "INVALID_ARGUMENT",
        "details": [
            {
                "@type": "type.googleapis.com/google.rpc.ErrorInfo",
                "reason": "INVALID_JSON_PROTO_MESSAGE",
                "domain": "pubsub.googleapis.com",
                "metadata": {
                    "message": "Message failed schema validation",
                    "revisionInfo": "Could not validate message with any schema revision for schema: projects/88888888/schemas/my-topic-schema, last checked revision: revision_id=xxxxx failed with status: Invalid data in message: Could not parse JSON message."
                }
            }
        ]
    }
}

the schema can be changed to avro or proto3 or anything if it can solve the issue (but must fulfill the BQ table criteria)

Thank You

It looks like you are send a message that only puts fields in the attributes of the Pub/Sub message rather than sending the data portion of the message as JSON. A schema is not used to validate the attributes; it validates the data field. You would also need to base64-encode the data portion.

So if you want to send the JSON object

{
  "id": "ID123",
  "profile_id": "123",
  "address": "somewhere in the world",
  "date": "2023-07-20",
}

you would need to send the curl command:

curl --location --request POST 'https://pubsub.googleapis.com/v1/projects/my-project/topics/my-topic:publish' \
--header 'Authorization: Bearer myAccessToken' \
--header 'Content-Type: application/json' \
--data-raw '{
    "messages": [
        {
            "data": "ewogICJpZCI6ICJJRDEyMyIsCiAgInByb2ZpbGVfaWQiOiAiMTIzIiwKICAiYWRkcmVzcyI6ICJzb21ld2hlcmUgaW4gdGhlIHdvcmxkIiwKICAiZGF0ZSI6ICIyMDIzLTA3LTIwIiwKfQ=="
        }
    ]
}'