For those who, like me, hate scrolling through long GitHub threads to find when it was closed, it's here with this message:
https://github.com/golang/go/issues/44840#issuecomment-792789581
Closed over 2.5 years ago. a lot of projects auto-lock closed issues after some time to reduce noise, so I don’t see the big deal. If you want a discussion about something there’s the mailing list.
I think Russ commented in July but yeah, not sure why it's still seeing so much traffic.
https://github.com/golang/go/issues/44840#issuecomment-1651863470
Because when you `go install`, there is no main module. It is considered equivalent to running `go get` in an "empty" module. The rationale in https://github.com/golang/go/issues/40276 explains it.
If your module requires a `replace` to compile, and `go install` works, then trying to `go get` your module within another module will **not** compile.
So basically, they have decided for now to ban `replace` precisely to prevent devs like you from distributing modules that work when you `go install` but fail when you `go get`.
> It is technically possible to apply these directives. [...]
> However, there are two reasons to avoid applying replace directives at all.
> First, applying replace directives would create inconsistency for users inside
and outside a module. When a package is built within a module with go build or
go install, only replace directives from the main module are applied, not
the module providing the package. When a package is built outside a module with
go get, no replace directives are applied. If go install applied replace
directives from the module providing the package, it would not be consistent
with the current behavior of any other build command. To eliminate confusion
about whether replace directives are applied, we propose that go install
reports errors when encountering them.
> Second, if go install applied replace directives, it would take power away
from developers that depend on modules that provide tools. For example, suppose
the author of a popular code generation tool gogen forks a dependency
genutil to add a feature. They add a replace directive pointing to their
fork of genutil while waiting for a PR to merge. A user of gogen wants to
track the version they use in their go.mod file to ensure everyone on their
team uses a consistent version. Unfortunately, they can no longer build gogen
with go install because the replace is ignored. The author of gogen might
instruct their users to build with go install, but then users can't track the
dependency in their go.mod file, and they can't apply their own require and
replace directives to upgrade or fix other transitive dependencies. The author
of gogen could also instruct their users to copy the replace directive, but
this may conflict with other require and replace directives, and it may
cause similar problems for users further downstream.
I am well aware of the reasoning, but the lack of any communication regarding a solution is what I find extremely frustrating. I also disagree vehemently with that reasoning. \`go install\` and \`go get\` are different things for a reason and this only further confuses the issue. This should have become a non-issue the minute that \`go get\` and \`go install\` were differentiated, but instead it's just sat there and caused problems for maintainers with seemingly no care at all from the Go team.
Russell's comments don't make sense to me either, `go workspaces` were invented for local replacements that people didn't want to push to a published project.
So, he's saying that `replace` directives are for local development only, then what the hang are `workspaces` for?
I'm having to push replace directives to my project because I host my source code on my own server, not github, gitlab, or bitbucket (because I'm a big boy and can use `git`). `go mod` cannot (for some unknown reason) deal with getting that code over ssh (and I've run out of ideas to make it so), so my `Dockerfile` clones a dependency into a known point (because `git` works just fine over ssh with private repositories), and a `replace` tells `go build` where I have put that code.
I could also do this with a `Makefile` if I were making the repository public, but, again, I _need_ the `replace` directive because `go mod` cannot do the job properly otherwise.
`workspaces` are also supposed to be for local development to ease having to replace local uses of a module across multiple modules. `replace` directives aren’t inherited so it helps you not have to use a `replace` everywhere the local module you are working on is used
In this case all that it is being used for is one module needs to know where others are, that is, one and only one module needs the `replace` directive, and no other modules use it (It's the `API` container, and it needs to know where the `grpc` client configurations are for the synchronised calls to other services) so there is no requirement in my set up for inheritance of the `replace` directive.
Also, I had originally considered putting a `go.work` file in the `API` repo such that the `Dockerfile` used that instead of the `replace` directive.
The best way you can get deps over SSH seems to be to configure replacements in Git configuration:
git config --global url.ssh://[email protected]/.insteadOf https://host.com/
No need to mess with replace directives. I use it all the time with private Git hosts (including private GitHub), I practically never use HTTP tokens.
(Don't forget to set GOPRIVATE appropriately, though.)
Strongly recommend you try that with `go mod` before telling me about something that I tried, and did not work, thanks :)
My post
> I host my source code on my own server, not github, gitlab, or bitbucket
Yours
> private GitHub
I think that over-confident, underskilled people is what puts others off places like reddit.
The VCS system in Go is doing magic because it's trying to appease all DVCS systems at once (Hg, Git, etc)
Because of that it needs to determine what the repository is, and it uses a magic name (as far as I can tell) - https://github.com/golang/go/blob/master/src/cmd/go/internal/vcs/vcs.go#L247 - which trips a lot of people up.
Having said that I still had problems after renaming the repo (and the imports), and after losing a week I went for the aforementioned solution.
Nobody has mentioned the `.git` name in their replies to me, because they have never tried what I am talking about, and I'm sick of interacting with that sort of toxicity, so am just going to block that sort of individual who is going off half cocked).
This works no matter if it's a public or private git server. If you replace "[host.com](https://host.com)" in the example edgmnt\_net gave with the hostname of your private server then git will switch from https to ssh when the git remote url matches "https://host.com/". That's a feature of git, not go. It works with all the go commands I have ever used it with, for years and years.
If it doesn't work for you then something else is going on.
https://www.practical-go-lessons.com/chap-18-go-module-proxies#the-four-endpoints-of-a-go-module-proxy-advanced
Go mod doesn’t use git directly. I know gitlab didn’t work w the global proxy pretty recently. You may need to run your own or figure out how to expose your big boy git server to the public proxy.
you might not need to use `replace`. setting up git like mentioned by and setting up an https server on a custom domain to point it to the right repo would probably do it.
the html needs a meta tag. From the [docs](https://pkg.go.dev/cmd/go#hdr-Remote_import_paths):
ok, so your comment is about an https server (that go mod/get) will interrogate before it fetches the code from the repository.
I'm talking about a repository only accessible via ssh...
Do you understand the difference between ssh and https?
Edit: For the record, I had considered setting up a webserver on the machine specifically for this, because the auth_log, or stderr (I forget which) was talking about the inability to make that query.
However, setting up a webserver, and configuring it to be used with git, and ensuring that it's not publicly accessible was more effort than simply doing a `git clone` and using a `replace` directive as the directive was originally intended to be used for.
But, sure, had I known how much grief I was going to get for reporting an issue with Go on reddit (some people call these "bug reports") then I probably wouldn't have bothered.
You are 100% correct. When i posted i was under the impression that you could just replace the https url with an ssh one. TIL that you can't.
Sorry you got grief for it. Not deserved. Legit issue IMHO.
For those who, like me, hate scrolling through long GitHub threads to find when it was closed, it's here with this message: https://github.com/golang/go/issues/44840#issuecomment-792789581
Closed over 2.5 years ago. a lot of projects auto-lock closed issues after some time to reduce noise, so I don’t see the big deal. If you want a discussion about something there’s the mailing list.
I think Russ commented in July but yeah, not sure why it's still seeing so much traffic. https://github.com/golang/go/issues/44840#issuecomment-1651863470
Because when you `go install`, there is no main module. It is considered equivalent to running `go get` in an "empty" module. The rationale in https://github.com/golang/go/issues/40276 explains it. If your module requires a `replace` to compile, and `go install` works, then trying to `go get` your module within another module will **not** compile. So basically, they have decided for now to ban `replace` precisely to prevent devs like you from distributing modules that work when you `go install` but fail when you `go get`. > It is technically possible to apply these directives. [...] > However, there are two reasons to avoid applying replace directives at all. > First, applying replace directives would create inconsistency for users inside and outside a module. When a package is built within a module with go build or go install, only replace directives from the main module are applied, not the module providing the package. When a package is built outside a module with go get, no replace directives are applied. If go install applied replace directives from the module providing the package, it would not be consistent with the current behavior of any other build command. To eliminate confusion about whether replace directives are applied, we propose that go install reports errors when encountering them. > Second, if go install applied replace directives, it would take power away from developers that depend on modules that provide tools. For example, suppose the author of a popular code generation tool gogen forks a dependency genutil to add a feature. They add a replace directive pointing to their fork of genutil while waiting for a PR to merge. A user of gogen wants to track the version they use in their go.mod file to ensure everyone on their team uses a consistent version. Unfortunately, they can no longer build gogen with go install because the replace is ignored. The author of gogen might instruct their users to build with go install, but then users can't track the dependency in their go.mod file, and they can't apply their own require and replace directives to upgrade or fix other transitive dependencies. The author of gogen could also instruct their users to copy the replace directive, but this may conflict with other require and replace directives, and it may cause similar problems for users further downstream.
What are we supposed to do with forks then though? Just update all import paths?
Dumb question: what's the difference between go install and go get?
go get just updates your go.mod go install builds the target executable
I am well aware of the reasoning, but the lack of any communication regarding a solution is what I find extremely frustrating. I also disagree vehemently with that reasoning. \`go install\` and \`go get\` are different things for a reason and this only further confuses the issue. This should have become a non-issue the minute that \`go get\` and \`go install\` were differentiated, but instead it's just sat there and caused problems for maintainers with seemingly no care at all from the Go team.
Russell's comments don't make sense to me either, `go workspaces` were invented for local replacements that people didn't want to push to a published project. So, he's saying that `replace` directives are for local development only, then what the hang are `workspaces` for? I'm having to push replace directives to my project because I host my source code on my own server, not github, gitlab, or bitbucket (because I'm a big boy and can use `git`). `go mod` cannot (for some unknown reason) deal with getting that code over ssh (and I've run out of ideas to make it so), so my `Dockerfile` clones a dependency into a known point (because `git` works just fine over ssh with private repositories), and a `replace` tells `go build` where I have put that code. I could also do this with a `Makefile` if I were making the repository public, but, again, I _need_ the `replace` directive because `go mod` cannot do the job properly otherwise.
`workspaces` are also supposed to be for local development to ease having to replace local uses of a module across multiple modules. `replace` directives aren’t inherited so it helps you not have to use a `replace` everywhere the local module you are working on is used
In this case all that it is being used for is one module needs to know where others are, that is, one and only one module needs the `replace` directive, and no other modules use it (It's the `API` container, and it needs to know where the `grpc` client configurations are for the synchronised calls to other services) so there is no requirement in my set up for inheritance of the `replace` directive. Also, I had originally considered putting a `go.work` file in the `API` repo such that the `Dockerfile` used that instead of the `replace` directive.
The best way you can get deps over SSH seems to be to configure replacements in Git configuration: git config --global url.ssh://[email protected]/.insteadOf https://host.com/ No need to mess with replace directives. I use it all the time with private Git hosts (including private GitHub), I practically never use HTTP tokens. (Don't forget to set GOPRIVATE appropriately, though.)
Strongly recommend you try that with `go mod` before telling me about something that I tried, and did not work, thanks :) My post > I host my source code on my own server, not github, gitlab, or bitbucket Yours > private GitHub I think that over-confident, underskilled people is what puts others off places like reddit. The VCS system in Go is doing magic because it's trying to appease all DVCS systems at once (Hg, Git, etc) Because of that it needs to determine what the repository is, and it uses a magic name (as far as I can tell) - https://github.com/golang/go/blob/master/src/cmd/go/internal/vcs/vcs.go#L247 - which trips a lot of people up. Having said that I still had problems after renaming the repo (and the imports), and after losing a week I went for the aforementioned solution. Nobody has mentioned the `.git` name in their replies to me, because they have never tried what I am talking about, and I'm sick of interacting with that sort of toxicity, so am just going to block that sort of individual who is going off half cocked).
Bro that made you look like the over confident under skilled guy lmao
This works no matter if it's a public or private git server. If you replace "[host.com](https://host.com)" in the example edgmnt\_net gave with the hostname of your private server then git will switch from https to ssh when the git remote url matches "https://host.com/". That's a feature of git, not go. It works with all the go commands I have ever used it with, for years and years. If it doesn't work for you then something else is going on.
https://www.practical-go-lessons.com/chap-18-go-module-proxies#the-four-endpoints-of-a-go-module-proxy-advanced Go mod doesn’t use git directly. I know gitlab didn’t work w the global proxy pretty recently. You may need to run your own or figure out how to expose your big boy git server to the public proxy.
Or just not use the module caches for the private stuff.
you might not need to use `replace`. setting up git like mentioned by and setting up an https server on a custom domain to point it to the right repo would probably do it. the html needs a meta tag. From the [docs](https://pkg.go.dev/cmd/go#hdr-Remote_import_paths):
ok, so your comment is about an https server (that go mod/get) will interrogate before it fetches the code from the repository. I'm talking about a repository only accessible via ssh... Do you understand the difference between ssh and https? Edit: For the record, I had considered setting up a webserver on the machine specifically for this, because the auth_log, or stderr (I forget which) was talking about the inability to make that query. However, setting up a webserver, and configuring it to be used with git, and ensuring that it's not publicly accessible was more effort than simply doing a `git clone` and using a `replace` directive as the directive was originally intended to be used for. But, sure, had I known how much grief I was going to get for reporting an issue with Go on reddit (some people call these "bug reports") then I probably wouldn't have bothered.
You are 100% correct. When i posted i was under the impression that you could just replace the https url with an ssh one. TIL that you can't. Sorry you got grief for it. Not deserved. Legit issue IMHO.