Proto Best Practices — references to googleapis/googleapis common objects

[software-artificer]

👋🏻 hey folks!

Not sure if this is an actual issue or just missing documentation, so feel free to point me out to the right place.

I am wondering how do people deal with "common" protobuf messages that are referenced in the "Proto Best Practices" document but are not shipped with the protoc compiler. I am talking about google.types.* specifically here, which are located in the googleapis/googleapis. Feels a bit strange relying on someone's else API schemas.

• Why aren't these messages part of the well-known types shipped with the protobuf?
• How do people generally make them available in the CI and to the developers? Copy? Clone? Use googleapis/api-common-protos?
• Does it make sense to include these into the protocolbuffers project itself?

Thanks in advance!

[Logofile]

Thanks for submitting your question. Can you point me to an example of the common messages referenced in the Proto Best Practices topic? I'll work to get an answer to you--and potentially update the docs with it, as well--once I better understand the question. Thanks!

[software-artificer]

@Logofile thanks for your prompt response. And of course, should have done so ahead of time, sorry. Here's the excerpt pointing to googleapis:

protocolbuffers.github.io/content/best-practices/dos-donts.md

Lines 147 to 169 in 354da49

	* [`interval`](https://github.com/googleapis/googleapis/blob/master/google/type/interval.proto)
	is a time interval independent of time zone or calendar (for example,
	2017-01-15T01:30:15.01Z - 2017-01-16T02:30:15.01Z).
	* [`date`](https://github.com/googleapis/googleapis/blob/master/google/type/date.proto)
	is a whole calendar date (for example, 2005-09-19).
	* [`month`](https://github.com/googleapis/googleapis/blob/master/google/type/month.proto)
	is a month of year (for example, April).
	* [`dayofweek`](https://github.com/googleapis/googleapis/blob/master/google/type/dayofweek.proto)
	is a day of week (for example, Monday).
	* [`timeofday`](https://github.com/googleapis/googleapis/blob/master/google/type/timeofday.proto)
	is a time of day (for example, 10:42:23).
	* [`field_mask`](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/field_mask.proto)
	is a set of symbolic field paths (for example, f.b.d).
	* [`postal_address`](https://github.com/googleapis/googleapis/blob/master/google/type/postal_address.proto)
	is a postal address (for example, 1600 Amphitheatre Parkway Mountain View,
	CA 94043 USA).
	* [`money`](https://github.com/googleapis/googleapis/blob/master/google/type/money.proto)
	is an amount of money with its currency type (for example, 42 USD).
	* [`latlng`](https://github.com/googleapis/googleapis/blob/master/google/type/latlng.proto)
	is a latitude/longitude pair (for example, 37.386051 latitude and
	-122.083855 longitude).
	* [`color`](https://github.com/googleapis/googleapis/blob/master/google/type/color.proto)
	is a color in the RGBA color space.

All of the links there point to googleapis/googleapis repository, with the excepiton of field_mask which is shipped with protobuf includes.

[Logofile]

🤦 I don't know why I didn't see that section when I reviewed it. I blame the fact that it was later I the day on a Friday.

I'm not aware of the reason for the separation of the content, but since that's a code issue rather than a documentation issue, I recommend adding an issue in https://github.com/protocolbuffers/protobuf/issues. If you'd like to see a CI/CD best practice paragraph added to the documentation, let's leave this issue open to run alongside the code repository issue.

[software-artificer]

@Logofile I know the feel and thank you very much for your guidance.

I went ahead and created the issue in the protocolbuffers/protobuf. Let's keep this one open until we get any more clarification there. If the common types will be included in the protoc package this could safely be closed. If not, then we can figure out how to properly document the best practices of fetching these proto files and making them available in CI/CD and development environments.

[Logofile]

It sounds like we're in agreement. 😊

[mkruskal-google]

@Logofile Closing protocolbuffers/protobuf#21849 and punting this back to you as a documentation bug 🙂. See my response there, it seems really unlikely we'd ever add new WKTs given the problems we've had with them

[Logofile]

Thanks! I'l work with @software-artificer in the other issues to get the docs updated.

[Logofile]

Oops. I got the issues mixed up. This one is the one to keep open. 🤦 Reopening....

[software-artificer]

Considering we got quite strong "bad vibes" for well-known and common types as the challenges that are associated with them, how do you think we should go about documenting this?

• Should we keep WKTs as recommended, provided they're there to stay, but suggest that people get inspired by the Google's API design for other types? I.e. get the idea of how people design types for particular use cases and just to your needs kind-of style...
• Or should we just recommend pulling Google's types into their projects and document how to do this? Which on its own presents the challenge of them not being updated often, as far as I can tell they are still not using editions. But also potentially introducing breaking changes without consumers being aware, because Google is free to iterate on their API however they feel comfortable.

I am leaning towards just saying "use WKTs and get inspired by some of the designs from Google", but I am of course will follow the your guidance here @Logofile as a mainatiner.

I have recently ran into an issue where I updated from proto3 to editions and this caused breaking changes in some generated code, despite not having any changes to the messages themselves, so even something like this could be a problem...

[Logofile]

I'm so glad to have your collaboration on this. I primarily see how Googlers use protobufs, so having an OSS perspective is really helpful.

I prefer your second option for a couple of reasons:

• We don't change implementation details about our WKTs often at all, and that's by design. They should be well-thought-out and fairly static... that's part of the appeal.
• We notify protobuf users about upcoming breaking changes (see https://protobuf.dev/news for a collection of them). So if developers want to stay on top of changes, they have a mechanism to do so.

Related to your recent experience with moving from proto3 to editions, I'm curious what breakage you ran into. Did you create an issue for it? We're working on tooling that will automate the transition from proto2/proto3 to editions to hopefully eliminate (or at least greatly reduce) any implementation-specific sharp corners.

[software-artificer]

Thank you 🙇‍♂ and I appreciate your support on this.

Let me clarify things a bit so we are definitely on the same page.

When I am saying WKTs I specifically mean the well-known types that are shipped with the protobuf package. I believe that we both are in agreement that they should stay there and be used, recommended and documented. These include things like google.protobuf.StringValue or google.protobuf.BoolValue, etc.

When I am talking about "Google's types" I specifically mean ones from googleapis/googleapis located in the google.type package — they aren't part of the protobuf.dev and aren't shipped with the compiler. If Google APIs were to change and evolve, I presume they won't be covered by the breaking changes in the protobuf.dev news/changelog, hence I was inclined to document them differently perhaps, as in: "feel free to get inspired by their design, but make your own". Does this make sense?

---

When it comes to the breaking changes that I have recently observed in one of our projects it was when we flipped from proto3 to editions and got some generated code in Go changed from using concrete values to using pointers.

[Logofile]

Ah, thanks for that clarification! I didn't realize that we separate our Well-Known Types from other types that are provided by Google, but under a different package. So you're not asking about the WKT (which are available when you install protobuf), but rather asking that we document how to retrieve non-WKT Google types? If I'm still off, do please correct me. 😄

I will check with the engineering team to find out what our recommendations are and update this topic with that information.