Skip to content

Kotlin: Implement JvmOverloads annotation#9811

Merged
smowton merged 3 commits intogithub:mainfrom
smowton:smowton/feature/kotlin-jvmoverloads-annotation
Oct 4, 2022
Merged

Kotlin: Implement JvmOverloads annotation#9811
smowton merged 3 commits intogithub:mainfrom
smowton:smowton/feature/kotlin-jvmoverloads-annotation

Conversation

@smowton
Copy link
Copy Markdown
Contributor

@smowton smowton commented Jul 12, 2022

This generates functions that omit parameters with default values, rightmost first, such that Java can achieve a similar experience to Kotlin (which represents calls internally as if the default was supplied explicitly, and/or uses a $default method that supplies the needed arguments).

A complication: combining JvmOverloads with JvmStatic means that both the companion object and the surrounding class get overloads.

@smowton smowton requested review from a team as code owners July 12, 2022 18:45
@smowton smowton added the no-change-note-required This PR does not need a change note label Jul 12, 2022
maybeParentId,
getFunctionShortName(f).nameInDB,
f.valueParameters,
maybeParameterList ?: f.valueParameters,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add an explanation of when/why maybeParameterList is passed in to the function's comment please?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@smowton smowton force-pushed the smowton/feature/kotlin-jvmoverloads-annotation branch from ef18a20 to ecb473b Compare September 21, 2022 20:15
Copy link
Copy Markdown
Contributor

@tamasvajk tamasvajk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some minor comments. The PR looks generally okay, but some tests are still failing.

Also, I think we'll have problems with introducing the new parameters with type IrValueParameterWithOverriddenIndex. They have incorrect parents, so we might generate incorrect label for them. I haven't checked it, but in case a parameter is accessed in a default argument value, extractExpressionExpr doesn't have the overloadId:

fun fn(s0: String = "", s1: String = fn(s0, ""), s2: String = "") = ""

Maybe storing the overloadId in IrValueParameterWithOverriddenIndex would help.

import java.util.*
import kotlin.collections.ArrayList

fun emptyTypeParameterMap() = HashMap<IrDeclarationParent, Label<out DbClassorinterfaceorcallable>>()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpicking: HashMap<IrDeclarationParent, Label<out DbClassorinterfaceorcallable>>() is not an emptyTypeParameterMap, but rather an emptyTypeParameterParentMap, isn't it?

@smowton smowton force-pushed the smowton/feature/kotlin-jvmoverloads-annotation branch from d635254 to e4f73bf Compare September 26, 2022 08:18
@smowton
Copy link
Copy Markdown
Contributor Author

smowton commented Sep 26, 2022

@tamasvajk now that I've had to use a piece of background state to cope with type variable references, I've changed the strategy for handling value parameters to use the same mechanism instead of creating a proxy class that wraps IrValueParameter. I've also converted the remaining integration test into a unit test, and added a test for the case you pointed out where one parameter refers to another one.

Copy link
Copy Markdown
Contributor

@tamasvajk tamasvajk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some questions for discussion.

}

private inner class DeclarationStackAdjuster(declaration: IrDeclaration): Closeable {
private inner class DeclarationStackAdjuster(val declaration: IrDeclaration, val overriddenAttributes: OverriddenFunctionAttributes? = null): Closeable {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@igfoo You wanted to get rid of the declaration stack completely. If that's still the plan, then we might need to look for another solution here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't worry about that for now. We can revisit this if/when that happens.

}

private inner class DeclarationStackAdjuster(declaration: IrDeclaration): Closeable {
private inner class DeclarationStackAdjuster(val declaration: IrDeclaration, val overriddenAttributes: OverriddenFunctionAttributes? = null): Closeable {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would probably be cleaner to have another constructor, which takes an IrFunction and an OverriddenFunctionAttributes.

val externalClassExtractor: ExternalDeclExtractor,
val primitiveTypeMapping: PrimitiveTypeMapping,
val pluginContext: IrPluginContext,
val overriddenFunctionAttributes : HashMap<IrDeclaration, OverriddenFunctionAttributes>,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't this be an HashMap<IrFunction, ... ?

Comment on lines +1383 to +1454
(it as? IrDeclaration)?.let { decl ->
overriddenFunctionAttributes[decl]?.id
} ?:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this and all other overriddenFunctionAttributes uses in KotlinUsesExtractor assume that when the access is made, we're extracting the given function with overrides? So in this case, are we sure that getTypeParameterParentLabel is only called when overriddenFunctionAttributes[decl] is being extracted?

I think, you might be right that getTypeParameterParentLabel and getValueParameterLabel are only called within forceExtractFunction and extractGeneratedOverload, but it feels somewhat fragile. Isn't this the same as (this as KotlinFileExtractor).declarationStack.peek().overriddenFunctionAttributes, which doesn't make sense at the moment, because the attributes are not on the stack, but if they were, we would consider this dangerous because of the cast.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For value parameters: I'd expect this could be seen from inside capturing local declarations and classes too. Something like

fun f(x: Int, y: Int = x.let {
  fun captures() = it
  captures()
})

For type parameters: these definitely can be seen from elsewhere, principally when extracting instantiated generic types that occur in the signature of a method with overloads. So when we extract fun <T> f(x: List<T>, y: T? = null) then we need to generate List<T> with T = f/1::T as well as the case with T = f/2::T that gets created for the "normal" version of f. That's why the overridden functions map gets passed to child instances of KotlinUses/FileExtractor.

}

private inner class DeclarationStackAdjuster(declaration: IrDeclaration): Closeable {
private inner class DeclarationStackAdjuster(val declaration: IrDeclaration, val overriddenAttributes: OverriddenFunctionAttributes? = null): Closeable {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it make more sense to push (declaration, overriddenAttributes) onto the stack? That way we wouldn't have to maintain two sets of states (overriddenFunctionAttributes and declarationStack)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I separated them because I needed the function override state to be propagated when extracting generic type instantiations as mentioned above and didn't want to get into the decl stack being non-empty on completing extraction of that type, but I can keep them together if that's preferred at the slight cost of searching the stack.

Comment on lines +1523 to +1598
val overriddenParentAttributes = (declarationParent as? IrDeclaration)?.let {
overriddenFunctionAttributes[it]
}
val parentId = parent ?: overriddenParentAttributes?.id ?: useDeclarationParent(declarationParent, false)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this work with recursive calls too? overriddenFunctionAttributes only has a reference to one overload at a time. Could we need multiple at any time? For example when generating the overload f(String,String) would we need access to both f(String,String) and f(String)?

object X {
    @JvmStatic
    @JvmOverloads
    fun f(s0: String = f("", ""), s1: String = f(""), s2: String = "") = ""
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Answering this to myself: I think this question doesn't make sense. We're accessing parameters here, I don't think it's possible to access parameters of multiple overloads, we can at most access parameters that are declared earlier in the parameter list, and those will come from the current declaration.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these are ok because the overridden attributes only affect value and type parameter refs, whereas these two are referring to the function as a callee

@smowton
Copy link
Copy Markdown
Contributor Author

smowton commented Sep 26, 2022

@tamasvajk I've adopted your suggestion to store the overridden attributes in the declaration stack

tamasvajk
tamasvajk previously approved these changes Sep 28, 2022
Copy link
Copy Markdown
Contributor

@tamasvajk tamasvajk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added some refactoring ideas, feel free to ignore them.

I think the PR is in a mergeable state, we know there are some missing cases, which will be covered in upcoming PRs. We could also apply the refactorings later.

val globalExtensionState: KotlinExtractorGlobalState
) {

class DeclarationStack {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find it somewhat odd that the declaration stack is in the UsesExtractor. In my mind it belongs to the file extractor because that's where we extract the declarations.

fun getValueParameterLabel(vp: IrValueParameter, parent: Label<out DbCallable>?): String {
val declarationParent = vp.parent
val parentId = parent ?: useDeclarationParent(declarationParent, false)
val overriddenParentAttributes = (declarationParent as? IrFunction)?.let {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we kept the declaration stack in the file extractor, we could use the below:

val overriddenParentAttributes = if (this is KotlinFileExtractor && this.filePath == vp.file.path && declarationParent is IrFunction) {
  this.declarationStack.findOverriddenAttributes(declarationParent)
} else {
  null
}

I find this more explicit in what our expectations are: the overridden attributes should only be used if we're inside the extraction of the corresponding function declaration.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know if this is a wrong assumption.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vp.file.path can throw, so the correct check would be vp.fileOrNull?.path. or declarationParent .fileOrNull?.path.

I think adding the check would make it explicit that we only expect to find anything by findOverriddenAttributes if the files match. It would only make the intent more explicit, but it wouldn't change the behaviour. Also, performance wise having the check or not probably doesn't matter, because both fileOrNull and findOverriddenAttributes walks a parent-child declaration list linearly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok done

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same should be done in getTypeParameterParentLabel just to keep the symmetry between the two cases.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not done, because this is the case where we do need to look for type-parameter substitution happening up the stack, even in a different file -- for example, due to generic type specialisations transitively required from a synthetic function's parameter or return types.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I always forget about this special case.

@smowton smowton force-pushed the smowton/feature/kotlin-jvmoverloads-annotation branch from cb1289d to 5e2c607 Compare October 3, 2022 14:28
This generates functions that omit parameters with default values, rightmost first, such that Java can achieve a similar experience to Kotlin (which represents calls internally as if the default was supplied explicitly, and/or uses a $default method that supplies the needed arguments).

A complication: combining JvmOverloads with JvmStatic means that both the companion object and the surrounding class get overloads.
Copy link
Copy Markdown
Contributor

@tamasvajk tamasvajk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some minor comments, otherwise it looks good to me.

Comment on lines +131 to +133
if (this is KotlinFileExtractor)
this.declarationStack
else
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed on Slack, I don't think we should reuse the declaration stack if the filePath doesn't match. I think we won't have any trouble by reusing it, at least not currently, but this means that we can end up with declaration stacks that contain declarations from multiple files, and therefore be in a state where the declarations in the stack do not contain each other in reality.

As an example: withFileOfClass is called from addClassLabel, which calls extractNonPrivateMemberPrototypes on the returned KotlinFileExtractor, and it internally calls extractFunction which does modify the declaration stack.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, "be in a state where the declarations in the stack do not contain each other in reality" is probably also true if the fileClass check holds. But in this case, at least the declarations in the stack would be from the same file.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This actually does need to be the same stack for the case of type variables -- for example, when extracting the List<T> (and its downstream types, Iterator<T> etc) in the context of synthetic fun <A> f(a: List<A>) related to the real function fun <B> f(a: List<B>, b: Int = 0), we need to substitute B -> A everywhere, and the stack of outstanding function extractions is used to find the enclosing extraction of the relevant function declaring the type variable.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks for the explanation.

fun getValueParameterLabel(vp: IrValueParameter, parent: Label<out DbCallable>?): String {
val declarationParent = vp.parent
val parentId = parent ?: useDeclarationParent(declarationParent, false)
val overriddenParentAttributes = (declarationParent as? IrFunction)?.let {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vp.file.path can throw, so the correct check would be vp.fileOrNull?.path. or declarationParent .fileOrNull?.path.

I think adding the check would make it explicit that we only expect to find anything by findOverriddenAttributes if the files match. It would only make the intent more explicit, but it wouldn't change the behaviour. Also, performance wise having the check or not probably doesn't matter, because both fileOrNull and findOverriddenAttributes walks a parent-child declaration list linearly.

@smowton smowton merged commit e29be41 into github:main Oct 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Java Kotlin no-change-note-required This PR does not need a change note

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants