I think you definitely need to approach a 100k line project differently to smaller code (for example declare types more often, static type at least your interfaces and methods, etc) - but it is completely possible to write Groovy in a style that scales to that size and is still maintainable. After all, it can look almost exactly like Java.
On the other hand, it's impossible to scale Java down to the point where it would be sensible choice for what would otherwise be a bash script.
I think Gradle is mostly implemented in Java. I wouldn't think its speed is limited by the language of the DSL. Although I would guess startup time is longer as it definitely does have to load up groovy in the classpath and initialize all its classes.