If you use Google Cloud and you use Java for your projects then Google Cloud Storage is a great place to host your teams artifacts.
It is easy to setup and pretty cheap. Also it is much simpler than setting one of the existing repository options (jfrog, nexus, archiva etc) if you are not particularly interested in their features.
To get started you need to specify a maven wagon which supports google cloud storage.
We will use the Google storage wagon.
Let’s get started by creating a maven project
mvn archetype:generate -DgroupId=com.test.apps -DartifactId=GoogleWagonTest -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false
We are going to add a simple service.
package com.test.apps;
public class HelloService {
public String sayHello() {
return "Hello";
}
}
Then we are going to add the maven wagon which will upload and fetch our binaries to the google cloud storage.
<build>
<extensions>
<extension>
<groupId>com.gkatzioura.maven.cloud</groupId>
<artifactId>google-storage-wagon</artifactId>
<version>1.0</version>
</extension>
</extensions>
</build>
Then we shall create the google cloud storage bucket which will host our artifacts.
Our bucket shall be called mavenrepository
Now that we have set up our bucket in google we shall set the distribution management on our maven project.
<distributionManagement>
<snapshotRepository>
<id>my-repo-bucket-snapshot</id>
<url>gs://mavenrepository/snapshot</url>
</snapshotRepository>
<repository>
<id>my-repo-bucket-release</id>
<url>gs://mavenrepository/release</url>
</repository>
</distributionManagement>
From the maven documentation
Where as the repositories element specifies in the POM the location and manner in which Maven may download remote artifacts for use by the current project, distributionManagement specifies where (and how) this project will get to a remote repository when it is deployed. The repository elements will be used for snapshot distribution if the snapshotRepository is not defined.
The next step is the most crucial and this has to to do with authenticating to google cloud.
You need to have the gcloud command line setup in your system and you must issue a login
‘gcloud auth login –brief’ with an account that has access to the bucket we created previously.
The other way is to use the GOOGLE_APPLICATION_CREDENTIALS environment variable. You can use this GOOGLE_APPLICATION_CREDENTIALS in order to set the path to your google application credentials file.
The credentials file should also be able to access the bucket we created previously.
And now the easiest part which is deploying.
mvn deploy
Now since your artifact has been deployed you can use it in another repo by specifying your repository and your wagon.
<repositories>
<repository>
<id>my-repo-bucket-snapshot</id>
<url>gs://mavenrepository/snapshot</url>
</repository>
<repository>
<id>my-repo-bucket-release</id>
<url>gs://mavenrepository/release</url>
</repository>
</repositories>
<build>
<extensions>
<extension>
<groupId>com.gkatzioura.maven.cloud</groupId>
<artifactId>google-storage-wagon</artifactId>
<version>1.0</version>
</extension>
</extensions>
</build>
That’s it! Next thing you know, your artifact will be downloaded by maven through google cloud storage and used as a dependency in your new project.
Emmanouil Gkatziouras 
Hey Emmanouil, Thanks for this helpful article. When you talk about adding the maven wagon, are you adding this to the pom.xml for the project? Or in the Maven settings? same question for distribution management. I am also having trouble seeing how the GoogleStorageWagon code from github comes into play.
Greetings!
You shall add them at the pom.xml of the project.
You don’t need to use the github code since it is uploaded on the maven central.
Ah, I see! Thanks a bunch.
Did you get this working at all? I’m stuck at specifying the Google Cloud Storage project using the wagon.gs.project.[repository id] property. I have this set, but it always tries to access my bucket at a project id of 455054200496 which I dont even know where that number is coming from.
I was able to get it working, the issue was with the permissions. It requires storage.buckets.list which is not obvious since the bucket is specified as part of the configuration, I dont know why it needs list permissions
I think I am running into the same error as you. How did you specify these permissions?
I run into the following exception
“`
Could not establish connection with google cloud
com.google.cloud.storage.StorageException: Anonymous caller does not have storage.buckets.list access to project 116955912135.
“`
To clarify, my account does have access to google cloud, and I can run `gcloud` commands from the terminal.
I had to create a new role with the permissions and add my service account user to the role. None of the default craator/view roles container the permission to list buckets
please check my comment below, it fixed this issue for me
“gcloud auth login” didn’t help.
A better way to authenticate is to
1. create a service account in GCP + create & download the access_key file for the service account
2. make sure(or assign) the service account has the “Storage Object Admin Viewer” permission
3. on your local machine, gcloud auth activate-service-account –key-file=
4. verify that the service account has permission to list buckets – run : gsutil ls
5. finally, “mvn deploy” should be successful.
Hi Emmanouil, thanks for the article this has been really useful. Do you know if it is possible to use the Wagon to store archetypes and then generate from them? I’ve not been able to find information about how I can to this?
Thanks
Dan
I have been able to manage this and put my solution on my Stack Overflow question. https://stackoverflow.com/questions/52625810/store-maven-archetype-in-google-cloud-storage
This is awesome Dan! Thank you for doing so!
Sorry on being late! I was preparing the 1.2 release
Hi, while this works great i’m getting checksum validation failed when downloading from gcs, have you seen this?
Unfortunately I cannot fix this. It has to do with the google java sdk forcing you to upload a file without having an access to the InputStream and thus this problem arises.
I have the same issue, did you managed to solve it?
The issue was fixed by upgrading ‘google-storage-wagon’ version to 1.8.
How to upload artifact and in which format into Google Cloud Storage?
I am getting this error. Can you provide the solution. I already have issued a login by using this command ‘gcloud auth login –brief’ but still getting below error.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy (default-deploy) on project altaf: Failed to deploy artifacts: Could not transfer artifact com.altaf:altaf:jar:1.0 from/to my-repo-bucket-release (gs://gupshupsms/release): Please configure you google cloud account by logging using gcloud and specify a default project -> [Help 1]
Check the section
“The next step is the most crucial and this has to to do with authenticating to google cloud.”
I have authenticated in both the way.
1. I already login with gcloud.
2. I created one service account and downloaded JSON file then set the environment path to GOOGLE_APPLICATION_CREDENTIALS
But still getting the same problem.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy (default-deploy) on project altaf: Failed to deploy artifacts: Could not transfer artifact com.altaf:altaf:jar:1.0 from/to my-repo-bucket-release (gs://gupshupsms/release): Please configure you google cloud account by logging using gcloud and specify a default project -> [Help 1]
I request you to please create a YouTube video on this topic. It will be very helpful to me.
To fixed the authentication error executing: gcloud auth application-default login
For reference check https://stackoverflow.com/a/53505938
I fixed the authentication error executing: gcloud auth application-default login
For reference check https://stackoverflow.com/a/53505938
It works well with the environment variable, but I couldn´t make it work without (on Mac, with version 1.5)
hello Emmanouil,
i am unable to download dependencies from gbucket.
push functionality is working.
“[ERROR] Unresolveable build extension: Plugin com.gkatzioura.maven.cloud:google-storage-wagon:1.0 or one of its dependencies could not be resolved: Failed to read artifact descriptor for com.gkatzioura.maven.cloud:google-storage-wagon:jar:1.0: Could not transfer artifact com.gkatzioura.maven.cloud:google-storage-wagon:pom:1.0 from/to mirrorId (gs://mavenrepository-a1/): Cannot access gs://mavenrepository-a1/ with type default using the available connector factories: BasicRepositoryConnectorFactory: Cannot access gs://mavenrepository-a1/ using the registered transporter factories: WagonTransporterFactory: java.util.NoSuchElementException”
The maven wagon is located on the maven repository.
Based on the logs you try to download the maven wagon through your google repository, this should not happen google-storage-wagon should be downloaded through the maven central repo.
I am trying to connect from eclipse/maven running on my windows laptop to maven repo on GCP storage. I am using GOOGLE_APPLICATION_CREDENTIALS env variable with a service account and appropriate permission.
I have NOT installed gcloud command line in my laptop.
Initially I was getting the “Anonymous caller” error. Once I corrected the key.json file the error stopped.
It proceeded to on to give a SSL certificate error. I downloaded the http://www.googleapis.com cert and added it to the trust store to the mvn params. It is still erroring out with no SSL cert found for domain.
Is it possible to connect without installing gcloud command line or am I missing something?
I am past the SSL issue now.
I am getting the following exception
Caused by: java.lang.IndexOutOfBoundsException
at java.base/java.nio.ByteBuffer.wrap(ByteBuffer.java:395)
at org.eclipse.aether.transport.wagon.WagonTransferListener.transferProgress(WagonTransferListener.java:64)
at com.gkatzioura.maven.cloud.listener.TransferListenerContainerImpl.lambda$fireTransferProgress$2(TransferListenerContainerImpl.java:75)
at java.base/java.util.Vector.forEach(Vector.java:1387)
at com.gkatzioura.maven.cloud.listener.TransferListenerContainerImpl.fireTransferProgress(TransferListenerContainerImpl.java:75)
at com.gkatzioura.maven.cloud.transfer.TransferProgressImpl.progress(TransferProgressImpl.java:36)
at com.gkatzioura.maven.cloud.transfer.TransferProgressFileInputStream.read(TransferProgressFileInputStream.java:65)
at com.vorstella.shade.com.google.api.client.util.ByteStreams.copy(ByteStreams.java:51)
at com.vorstella.shade.com.google.api.client.util.IOUtils.copy(IOUtils.java:94)
at com.vorstella.shade.com.google.api.client.http.AbstractInputStreamContent.writeTo(AbstractInputStreamContent.java:72)
at com.vorstella.shade.com.google.api.client.http.MultipartContent.writeTo(MultipartContent.java:107)
at com.vorstella.shade.com.google.api.client.http.GZipEncoding.encode(GZipEncoding.java:49)
at com.vorstella.shade.com.google.api.client.http.HttpEncodingStreamingContent.writeTo(HttpEncodingStreamingContent.java:51)
at com.vorstella.shade.com.google.api.client.http.javanet.NetHttpRequest$DefaultOutputWriter.write(NetHttpRequest.java:76)
at com.vorstella.shade.com.google.api.client.http.javanet.NetHttpRequest.writeContentToOutputStream(NetHttpRequest.java:156)
at com.vorstella.shade.com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:117)
at com.vorstella.shade.com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:84)
at com.vorstella.shade.com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1011)
at com.vorstella.shade.com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequestWithoutGZip(MediaHttpUploader.java:548)
at com.vorstella.shade.com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequest(MediaHttpUploader.java:565)
at com.vorstella.shade.com.google.api.client.googleapis.media.MediaHttpUploader.directUpload(MediaHttpUploader.java:360)
at com.vorstella.shade.com.google.api.client.googleapis.media.MediaHttpUploader.upload(MediaHttpUploader.java:334)
at com.vorstella.shade.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:508)
at com.vorstella.shade.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at com.vorstella.shade.com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:549)
at com.vorstella.shade.com.google.cloud.storage.spi.v1.HttpStorageRpc.create(HttpStorageRpc.java:303)
at com.vorstella.shade.com.google.cloud.storage.StorageImpl.create(StorageImpl.java:161)
at com.gkatzioura.maven.cloud.gcs.GoogleStorageRepository.put(GoogleStorageRepository.java:101)
I tried a simple hello world with
File file = new File ( “MyFilePath/artifactid-versionno.txt” );
Repository repository = new Repository(“MyRepo”, “gs://MyBucket/mavenrepository/snapshot”);
googleStorageWagon.connect(repository);
googleStorageWagon.put(file, “group.id/artifactid/vesionno/artifactid-versionno.txt”);
It is working fine.
But I am getting the above error if I do a mvn deploy
I am using JDK 11.
I was using 1.8 of google-storage-wagon. I downgraded to 1.7. It worked!!!
Not working for parent poms :
I created parent pom. Built and deployed it successfully. Now I created a project which extends the parent pom. The deploy fails with
org.eclipse.aether.transfer.NoTransporterException: java.util.NoSuchElementException
role: org.apache.maven.wagon.Wagon
roleHint: gs
Looks like the maven is not able to download the parent pom from maven central, it is not able to go to gcp since the pom is not fully loaded yet for it to identify the wagon extension.
Will check this out. Thank you!
Hi! To resolve this please create a file on .mvn/extensions.xml and add this
<extensions xmlns="http://maven.apache.org/EXTENSIONS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/EXTENSIONS/1.0.0 http://maven.apache.org/xsd/core-extensions-1.0.0.xsd"> <extension> <groupId>com.gkatzioura.maven.cloud</groupId> <artifactId>google-storage-wagon</artifactId> <version>2.1-SNAPSHOT</version> </extension> </extensions>Even after using the version 1.8 getting the error Checksum validation failed unable to retrieve the artifacts from the bucket .
Yes, it has to do with the java binary from google which executes the operations behind the scenes. It is a warning and is not causing any issues with regards to working as a proper wagon implementation.
I created an issue: https://github.com/gkatzioura/CloudStorageMaven/issues/53
Why does the plugin keep trying to connect Anonymously?
Please, help!
hey,
I have deployed my jar by following your blog https://egkatzioura.com/2018/04/09/host-your-maven-artifacts-using-google-cloud-storage/. But i want to access the same remote repo which is being hosted using Google cloud storage in a gradle project. If i try to do so my artifacts aren’t getting downloaded.
The issue is with the placement of dependency com.gkatzioura.maven.cloud .
It would be great if you help me here!
Thanks,
Shashank
Hi there, I’ve successfully used your wagon in the past. I’m not able to use it now due to “”.
There must be something legacy, such as old Google Cloud libraries, which don’t work well with new authentication mechanisms. I’m assuming this because I have tried all possible means of authenticating with gcloud and also the environment variable GOOGLE_APPLICATION_CREDENTIALS.
For your reference, I simply changed your artifact ID for this one and it worked without changing any authentication with gcloud: https://github.com/lahsivjar/gcp-storage-wagon
Hope this helps.
Regards
Facing the same issue.Did you find solution to this?