Context
I ran into this while working on adding native GCS multipart upload support to s3proxy (see gaul/s3proxy#76). We use setHost() to configure the GCS endpoint, and the MultipartUploadClient operations fail with DNS errors.
What happens
With setHost("https://storage.googleapis.com") (no trailing slash):
Expected: https://storage.googleapis.com/my-bucket?uploads
Actual: https://storage.googleapis.commy-bucket?uploads
^^^ missing slash!
This causes a DNS lookup failure since storage.googleapis.commy-bucket isn't a valid hostname.
Reproduction
HttpStorageOptions options = StorageOptions.http()
.setHost("https://storage.googleapis.com") // no trailing slash
.build();
MultipartUploadClient client = MultipartUploadClient.create(
MultipartUploadSettings.of(options));
// Fails with DNS error:
client.createMultipartUpload(...); // StorageException: storage.googleapis.commy-bucket
Workaround
Add a trailing slash to your host:
.setHost("https://storage.googleapis.com/") // note the trailing slash
Or just don't call setHost() at all if you're using the default GCS endpoint.
Root cause
In MultipartUploadHttpRequestManager.java, the URI templates don't start with /:
UriTemplate.expand(uri.toString(), "{bucket}/{key}?uploads", params, false);
Looking at the UriTemplate.expand implementation:
When the template doesn't start with /, it simply concatenates baseUrl + uriTemplate:
Why tests didn't catch this
The test harness (FakeHttpServer) always creates endpoints with a trailing slash:
URI endpoint = URI.create("http://localhost:" + port + "/");
So the bug only shows up when explicitly using setHost()
Context
I ran into this while working on adding native GCS multipart upload support to s3proxy (see gaul/s3proxy#76). We use
setHost()to configure the GCS endpoint, and theMultipartUploadClientoperations fail with DNS errors.What happens
With
setHost("https://storage.googleapis.com")(no trailing slash):This causes a DNS lookup failure since
storage.googleapis.commy-bucketisn't a valid hostname.Reproduction
Workaround
Add a trailing slash to your host:
Or just don't call
setHost()at all if you're using the default GCS endpoint.Root cause
In
MultipartUploadHttpRequestManager.java, the URI templates don't start with/:Looking at the
UriTemplate.expandimplementation:When the template doesn't start with
/, it simply concatenatesbaseUrl + uriTemplate:Why tests didn't catch this
The test harness (
FakeHttpServer) always creates endpoints with a trailing slash:So the bug only shows up when explicitly using
setHost()