Skip to content
This repository was archived by the owner on Oct 19, 2023. It is now read-only.

Added grpc extension. - #62

Merged
tmatsuo merged 2 commits into
masterfrom
grpc
Mar 9, 2016
Merged

tmatsuo merged 2 commits into
masterfrom
grpc

Conversation

@tmatsuo

@tmatsuo tmatsuo commented Mar 1, 2016

Copy link
Copy Markdown
Contributor

No description provided.

@JustinBeckwith

Copy link
Copy Markdown
Contributor

Well that didn't take long! LGTM

@tmatsuo

tmatsuo commented Mar 1, 2016

Copy link
Copy Markdown
Contributor Author

Yeah, I actually haven't build the docker image locally, so it is possible to fail building. Let's see the travis result.

@tmatsuo

tmatsuo commented Mar 2, 2016

Copy link
Copy Markdown
Contributor Author

Seems like we still need -beta.

@bshaffer

bshaffer commented Mar 2, 2016

Copy link
Copy Markdown
Contributor

hey! nice. Is the pecl command sufficient to install all gRPC's dependencies?

@bshaffer

bshaffer commented Mar 2, 2016

Copy link
Copy Markdown
Contributor

i.e. maybe we should include a gRPC call as part of the tests. To the URL shortener, for example.

@tmatsuo

tmatsuo commented Mar 2, 2016

Copy link
Copy Markdown
Contributor Author

Sure, it is nicer to have such tests. I'll try to add e2e tests accessing Cloud Pub/Sub (I don't think URL shortner has enabled gRPC).

@tmatsuo

tmatsuo commented Mar 3, 2016

Copy link
Copy Markdown
Contributor Author

I added an end to end test. Let's see the travis build result.

"repositories": [
{
"type": "vcs",
"url": "https://github.com/stanley-cheung/Protobuf-PHP"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is @stanley-cheung interested in submitting his changes datto/protobuf-php? Why are we using his fork?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We tried, but the upstream repo has not responded in a year: drslump/Protobuf-PHP#48

But because he has submited his package to Packagist, we can't submit our fork unless we change the repo name.

That was why I have to resort to this composer.json syntax to use his official package on Packagist in the require section but substitute the actual content with our fork.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this get pulled in implicitly as part of the composer requirements for grpc?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is true. By pulling grpc/grpc, you are also implicitly pulling datto/protobuf-php (with a substituted fork) yes.

Edit: sorry, I take that back. The composer.json file you should be looking at should be at the top level of the grpc repo: https://github.com/grpc/grpc/blob/master/composer.json. Packagist only look at composer.json at the top level of a repo. There we did not pull in datto/protobuf-php. I believe at that time we thought pulling in datto/protobuf-php should be left to user of grpc.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so there's no need for specifying datto/protofuf-php here. I will remove it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No problem, I added it back.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK gotcha. I advocate making it the default, since it will be very difficult for users to discover the need for them to override with your fork on their own.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I just tried this again yesterday. Even if the root level grpc/composer.json is going to pull in datto/protobuf-php and with my stanley-cheung override, as a user of grpc/grpc, you are not going to get the stanley-cheung override unless the user also puts that in their own composer.json. In other words, as a user of grpc/grpc, at the minimum, they still need to require grpc/grpc and add the stanley-cheung/protobuf-php override. But then at least they don't need to include datto/protobuf-php. It's not ideal but I can spell it out more in grpc's documentation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stanley-cheung Yes, we should add this to composer. Also, are you sure the override is required in the userland composer.json? If so, that seems like a bug in composer. And also if so, we could submit your fork as a new composer package, which would definitely fix the issue.

Requiring the users to override in composer.json rather than just call composer require grpc/grpc is not acceptable DX IMHO. We could fork to GoogleCloudPlatform and submit a new package from there.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I have tried this again and an end-user needs to do this override, and I agree that it's non-ideal. OTOH, this was supposed to be a temporary bridgegap before we have official PHP support in google/protobuf - it's being implemented now. So if we can wait for a few months, this will all go away and we will have the official solution.

@tmatsuo

tmatsuo commented Mar 3, 2016

Copy link
Copy Markdown
Contributor Author

Stopped the travis build for merging other PR quicker

@tmatsuo
tmatsuo force-pushed the grpc branch 2 times, most recently from ca6deee to 07c16b8 Compare March 3, 2016 22:21
@tmatsuo

tmatsuo commented Mar 4, 2016

Copy link
Copy Markdown
Contributor Author

Oooh, of course the auto generated PHP files don't pass our coding style check...

@tmatsuo
tmatsuo force-pushed the grpc branch 2 times, most recently from 9abd7f7 to 7f25466 Compare March 4, 2016 02:09
Comment thread testapps/php56_e2e/source-contexts.json Outdated
@@ -0,0 +1 @@
[{"labels": {"category": "remote_repo"}, "context": {"git": {"url": "git@github.com:GoogleCloudPlatform/php-docker.git", "revisionId": "9abd7f7f4adeaf400c774ed667553eec74a385c4"}}}, {"labels": {"category": "remote_repo"}, "context": {"git": {"url": "git@github.com:tmatsuo/php-docker.git", "revisionId": "9abd7f7f4adeaf400c774ed667553eec74a385c4"}}}] No newline at end of file

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey who are you man!

@tmatsuo

tmatsuo commented Mar 4, 2016

Copy link
Copy Markdown
Contributor Author

I fixed several things, but the test still fails:

2016/03/04 02:29:40 [error] 321#0: *11 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Call to a member function getTopics() on null in /app/web/grpc_pubsub.php on line 36" while reading response header from upstream, client: 172.17.0.5, server: , request: "GET /grpc_pubsub.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "tmp-dot-php-mvm-a.appspot.com"

Seems like ApplicationDefaultCredentials::getCredentials is not working. I'll investigate tomorrow.

@tmatsuo

tmatsuo commented Mar 4, 2016

Copy link
Copy Markdown
Contributor Author

I think I found a cause!!!

Invalid resource name given (name=projects/)

@tmatsuo

tmatsuo commented Mar 4, 2016

Copy link
Copy Markdown
Contributor Author

@bshaffer The test is green now. PTAL

Comment thread testapps/php56_e2e/web/grpc_pubsub.php Outdated
$call = $client->ListTopics(
$req,
[],
['call_credentials_callback' => 'updateAuthMetadataCallback']);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if we could use the credentials' getUpdateMetadataFunc function for this:

$auth_credentials = ApplicationDefaultCredentials::getCredentials(
     ['https://www.googleapis.com/auth/pubsub']
);
$call = $client->ListTopics(
     $req,
     [],
     ['call_credentials_callback' => $auth_credentials->getUpdateMetadataFunc())];

But it looks like since the first argument doesn't match up, this won't work. Maybe we should change the google/auth library to handle this? I assume that is what this function is meant for.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, so this callback 'updateAuthMetadataCallback' is meant to be seen in gRPC's perspective. It's as if the gRPC core library is saying, "I am ready for the auth credentials. Here's what we know about this call so far, contained in the $context variable (the most important one being $context->service_url, coz after load balancing, user may not know which is the host until now)". So with this $context variable, we ask the auth library for the auth header by calling $auth_credentials->updateMetadata passing $context->service_url to it.

To me these 2 callbacks don't need to lineup because the outer gRPC callback can do more than just Auth. I think it;s OK to leave this as is.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, thanks for the explanation. Is it a problem that the google/auth library isn't doing anything with the service_url?

Also, we could make this cleaner like this:

$auth_credentials = ApplicationDefaultCredentials::getCredentials(
     ['https://www.googleapis.com/auth/pubsub']
);
$call = $client->ListTopics(
     $req,
     [],
     ['call_credentials_callback' => function($context) use ($auth_credentials) {
         return $auth_credentials->updateMetadata([], $context->service_url);
     })];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the $service_url is used only in some xxxCredentials class, like ServiceAccountJwtAccessCredentials but not the others.

I like the inline function change.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh yes, I see that now. Very cool. SGTM, thanks @stanley-cheung!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@bshaffer

bshaffer commented Mar 4, 2016

Copy link
Copy Markdown
Contributor

LGTM

@tmatsuo

tmatsuo commented Mar 4, 2016

Copy link
Copy Markdown
Contributor Author

I just stopped the build, I will re-run later today.

@tmatsuo

tmatsuo commented Mar 9, 2016

Copy link
Copy Markdown
Contributor Author

Alright, merging.

tmatsuo pushed a commit that referenced this pull request Mar 9, 2016
@tmatsuo
tmatsuo merged commit 00570ba into master Mar 9, 2016
@tmatsuo
tmatsuo deleted the grpc branch March 9, 2016 23:33
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants