Compiler plugins are powerful metaprogramming tools that can greatly enhance
how you write code. Compiler plugins call compilers directly as libraries to
analyze and edit input programs. These plugins can also generate output for
various uses. For example, they can generate boilerplate code, and they can
even generate full implementations for specially-marked program elements,
such as Parcelable
. Plugins have a variety of other uses and can even be used
to implement and fine-tune features that are not provided directly in a
language.
While compiler plugins are powerful, this power comes at a price. To write even the simplest plugin, you need to have some compiler background knowledge, as well as a certain level of familiarity with the implementation details of your specific compiler. Another practical issue is that plugins are often closely tied to specific compiler versions, meaning you might need to update your plugin each time you want to support a newer version of the compiler.
Kotlin Symbol Processing (KSP) is an API that you can use to develop lightweight compiler plugins. KSP provides a simplified compiler plugin API that leverages the power of Kotlin while keeping the learning curve at a minimum. KSP is designed to hide compiler changes, minimizing maintenance efforts for processors that use it. KSP is designed not to be tied to the JVM so that it can be adapted to other platforms more easily in the future. KSP is also designed to minimize build times. For some processors, such as Glide, KSP reduces full compilation times by up to 25% when compared to KAPT.
KSP is itself implemented as a compiler plugin. There are prebuilt packages on Google's Maven repository that you can download and use without having to build the project yourself. For more information, see Try it out!
The KSP API processes Kotlin programs idiomatically. KSP understands Kotlin-specific features, such as extension functions, declaration-site variance, and local functions. KSP also models types explicitly and provides basic type checking, such as equivalence and assign-compatibility.
The API models Kotlin program structures at the symbol level according to Kotlin grammar. When KSP-based plugins process source programs, constructs like classes, class members, functions, and associated parameters are easily accessible for the processors, while things like if blocks and for loops are not.
Conceptually, KSP is similar to KType in Kotlin reflection. The API allows processors to navigate from class declarations to corresponding types with specific type arguments and vice-versa. Substituting type arguments, specifying variances, applying star projections, and marking nullabilities of types are also possible.
Another way to think of KSP is as a pre-processor framework of Kotlin programs. If we refer to KSP-based plugins as symbol processors, or simply processors, then the data flow in a compilation can be described in the following steps:
- Processors read and analyze source programs and resources.
- Processors generate code or other forms of output.
- The Kotlin compiler compiles the source programs together with the generated code.
Unlike a full-fledged compiler plugin, processors cannot modify the code. A compiler plugin that changes language semantics can sometimes be very confusing. KSP avoids that by treating the source programs as read-only.
Most processors navigate through the various program structures of the input source code. Before diving into usage of the API, let's look at how a file might look from KSP's point of view:
KSFile
packageName: KSName
fileName: String
annotations: List<KSAnnotation> (File annotations)
declarations: List<KSDeclaration>
KSClassDeclaration // class, interface, object
simpleName: KSName
qualifiedName: KSName
containingFile: String
typeParameters: KSTypeParameter
parentDeclaration: KSDeclaration
classKind: ClassKind
primaryConstructor: KSFunctionDeclaration
superTypes: List<KSTypeReference>
// contains inner classes, member functions, properties, etc.
declarations: List<KSDeclaration>
KSFunctionDeclaration // top level function
simpleName: KSName
qualifiedName: KSName
containingFile: String
typeParameters: KSTypeParameter
parentDeclaration: KSDeclaration
functionKind: FunctionKind
extensionReceiver: KSTypeReference?
returnType: KSTypeReference
parameters: List<KSValueParameter>
// contains local classes, local functions, local variables, etc.
declarations: List<KSDeclaration>
KSPropertyDeclaration // global variable
simpleName: KSName
qualifiedName: KSName
containingFile: String
typeParameters: KSTypeParameter
parentDeclaration: KSDeclaration
extensionReceiver: KSTypeReference?
type: KSTypeReference
getter: KSPropertyGetter
returnType: KSTypeReference
setter: KSPropertySetter
parameter: KSValueParameter
KSEnumEntryDeclaration
// same as KSClassDeclaration
This view lists common things that are declared in the file--classes, functions, properties, enums, and so on.
Every processor in KSP implements SymbolProcessor
:
interface SymbolProcessor {
fun init(options: Map<String, String>,
kotlinVersion: KotlinVersion,
codeGenerator: CodeGenerator,
logger: KSPLogger)
fun process(resolver: Resolver) // Let's focus on this
fun finish()
}
A Resolver
provides SymbolProcessor
with access to compiler details
such as symbols. A processor that finds all top-level functions and non-local functions in top-level
classes might look something like this:
class HelloFunctionFinderProcessor : SymbolProcessor() {
...
val functions = mutableListOf<String>()
val visitor = FindFunctionsVisitor()
override fun process(resolver: Resolver) {
resolver.getAllFiles().map { it.accept(visitor, Unit) }
}
inner class FindFunctionsVisitor : KSVisitorVoid() {
override fun visitClassDeclaration(classDeclaration: KSClassDeclaration, data: Unit) {
classDeclaration.getDeclaredFunctions().map { it.accept(this, Unit) }
}
override fun visitFunctionDeclaration(function: KSFunctionDeclaration, data: Unit) {
functions.add(function)
}
override fun visitFile(file: KSFile, data: Unit) {
file.declarations.map { it.accept(this, Unit) }
}
}
...
}
Get all member functions that are declared directly within a class:
fun KSClassDeclaration.getDeclaredFunctions(): List<KSFunctionDeclaration> {
return this.declarations.filterIsInstance<KSFunctionDeclaration>()
}
Determine whether a class or function is local to another function:
fun KSDeclaration.isLocal(): Boolean {
return this.parentDeclaration != null && this.parentDeclaration !is KSClassDeclaration
}
Determine whether a class member is visible to another:
fun KSDeclaration.isVisibleFrom(other: KSDeclaration): Boolean {
return when {
// locals are limited to lexical scope
this.isLocal() -> this.parentDeclaration == other
// file visibility or member
this.isPrivate() -> {
this.parentDeclaration == other.parentDeclaration
|| this.parentDeclaration == other
|| (
this.parentDeclaration == null
&& other.parentDeclaration == null
&& this.containingFile == other.containingFile
)
}
this.isPublic() -> true
this.isInternal() && other.containingFile != null && this.containingFile != null -> true
else -> false
}
}
// Find out suppressed names in a file annotation:
// @file:kotlin.Suppress("Example1", "Example2")
fun KSFile.suppressedNames(): List<String> {
val ignoredNames = mutableListOf<String>()
annotations.forEach {
if (it.shortName.asString() == "Suppress" && it.annotationType.resolve()?.declaration?.qualifiedName?.asString() == "kotlin.Suppress") {
it.arguments.forEach {
(it.value as List<String>).forEach { ignoredNames.add(it) }
}
}
}
return ignoredNames
}
The API definition can be found here.
The diagram below is an overview of how Kotlin is modeled in KSP:
In KSP, references to types are designed to be resolved by processors
explicitly (with a few exceptions) because most of the cost of the
underlying API implementation is in resolution. When a type is referenced,
such as KSFunctionDeclaration.returnType
or KSAnnotation.annotationType
,
it is always a KSTypeReference
, which is a KSReferenceElement
with
annotations and modifiers.
interface KSFunctionDeclaration : ... {
val returnType: KSTypeReference?
...
}
interface KSTypeReference : KSAnnotated, KSModifierListOwner {
val type: KSReferenceElement
}
A KSTypeReference
can be resolved to a KSType
, which refers to a type in
Kotlin's type system.
A KSTypeReference
has a KSReferenceElement
, which models Kotlin‘s program
structure: namely, how the reference is written. It corresponds to the
type
element in
Kotlin's grammar.
A KSReferenceElement
can be a KSClassifierReference
or
KSCallableReference
, which contains a lot of useful information without
the need for resolution. For example, KSClassifierReference
has
referencedName
, while KSCallableReference
has receiverType
,
functionArguments
, and returnType
.
If the original declaration referenced by a KSTypeReference
is needed,
it can usually be found by resolving to KSType
and accessing through
KSType.declaration
. Moving from where a type is mentioned to where its
class is defined looks like this:
KSTypeReference -> .resolve() -> KSType -> .declaration -> KSDeclaration
Type resolution is costly and is therefore made explicit. Some of the
information obtained from resolution is already available in
KSReferenceElement
. For example, KSClassifierReference.referencedName
can filter out a lot of elements that are not interesting. You should
resolve type only if you need specific information from KSDeclaration
or KSType
.
Note that a KSTypeReference
pointing to a function type has most of its
information in its element. Although it can be resolved to the family of
Function0
, Function1
, and so on, these resolutions don‘t bring any
more information than KSCallableReference
. One use case for resolving
function type references is dealing with the identity of the function's
prototype.
kotlinc
compiler plugins have access to almost everything from the compiler
and therefore have maximum power and flexibility. On the other hand, because
these plugins can potentially depend on anything in the compiler, they are
sensitive to compiler changes and need to be maintained frequently. These plugins
also require a deep understanding of kotlinc
’s implementation, so the learning
curve can be steep.
KSP aims to hide most compiler changes through a well-defined API, though major changes in compiler or even the Kotlin language might still require to be exposed to API users.
KSP tries to fulfill common use cases by providing an API that trades power for
simplicity. Its capability is a strict subset of a general kotlinc
plugin.
For example, while kotlinc
can examine expressions and statements and can even
modify code, KSP cannot.
While writing a kotlinc
plugin can be a lot of fun, it can also take a lot of
time. If you aren't in a position to learn kotlinc
’s implementation and do
not need to modify source code or read expressions, KSP might be a good fit.
KSP's API looks similar to kotlin.reflect
. The major difference between
them is that type references in KSP need to be resolved explicitly. This is
one of the reasons why the interfaces are not shared.
KAPT is a remarkable solution which makes a large amount of Java annotation processors work for Kotlin programs out-of-box. The major advantages of KSP over KAPT are improved build performance, not tied to JVM, a more idiomatic Kotlin API, and the ability to understand Kotlin-only symbols.
To run Java annotation processors unmodified, KAPT compiles Kotlin code
into Java stubs that retain information that Java annotation processors
care about. To create these stubs, KAPT needs to resolve all symbols in
the Kotlin program. The stub generation costs roughly 1/3 of a full
kotlinc
analysis and the same order of kotlinc
code-generation. For
many annotation processors, this is much longer than the time spent in
the processors themselves. For example, Glide looks at a very limited
number of classes with a predefined annotation, and its code generation
is fairly quick. Almost all of the build overhead resides in the stub
generation phase. Switching to KSP would immediately reduce the time
spent in the compiler by 25%.
For performance evaluation, we implemented a simplified version of Glide in KSP to make it generate code for the Tachiyomi project. While the total Kotlin compilation time of the project is 21.55 seconds on our test device, it took 8.67 seconds for KAPT to generate the code, and it took 1.15 seconds for our KSP implementation to generate the code.
Unlike KAPT, processors in KSP do not see input programs from Java's point
of view. The API is more natural to Kotlin, especially for Kotlin-specific
features such as top-level functions. Because KSP doesn't delegate to
javac
like KAPT, it doesn't assume JVM-specific behaviors and can be
used with other platforms potentially.
While KSP tries to be a simple solution for most common use cases, it has made several trade-offs compared to other plugin solutions. The following are not goals of KSP:
- Examining expression-level information of source code.
- Modifying source code.
- 100% compatibility with the Java Annotation Processing API.
We are also exploring several additional features. Note that these features are currently unavailable:
- IDE integration: Currently IDEs know nothing about the generated code.
- We are still investigating how to provide interoperability between processors.
If you are familiar with the Java Annotation Processing API, see the Java annotation processing to KSP reference for how to implement most functions and data structures that are provided by Java's API.
The API is still under development and is likely to change in the future.
Please do not use it in production yet. The purpose of this preview is
to get your feedback.
Please let us know what you think about KSP by filing a Github issue
or connecting with our team in the #ksp
channel in the
Kotlin Slack workspace!
Here are some planned features that have not yet been implemented:
- Passing options to processors in Gradle.
- Incremental processing.
- Improved support for reading Java source code.
- Make the IDE aware of the generated code.
Here's a sample processor that you can check out: https://github.com/google/ksp/releases/download/1.4.10-dev-experimental-20201120/playground-ksp-1.4.10-dev-experimental-20201120.zip
-
Create an empty gradle project.
-
Specify version
1.4.10
of the Kotlin plugin in the root project for use in other project modules.plugins { kotlin("jvm") version "1.4.10" apply false } buildscript { dependencies { classpath(kotlin("gradle-plugin", version = "1.4.10")) } }
-
Add a module for hosting the processor.
-
In the module's
build.gradle.kts
file, do the following:- Add
google()
to repositories so that Gradle can find our plugins. - Apply Kotlin plugin
- Add the KSP API to the
dependencies
block.
plugins { kotlin("jvm") } repositories { google() mavenCentral() } dependencies { implementation("com.google.devtools.ksp:symbol-processing-api:1.4.10-dev-experimental-20201120") }
- Add
-
The processor you're writing needs to implement
com.google.devtools.ksp.processing.SymbolProcessor
. Note the following:- Your main logic should be in the
process()
method. - Use
CodeGenerator
in theinit()
method for code generation. You can also save theCodeGenerator
instance for later use in eitherprocess()
orfinish()
. - Use
resolver.getSymbolsWithAnnotation()
to get the symbols you want to process, given the fully-qualified name of an annotation. - A common use case for KSP is to implement a customized visitor (interface
com.google.devtools.ksp.symbol.KSVisitor
) for operating on symbols. A simple template visitor iscom.google.devtools.ksp.symbol.KSDefaultVisitor
. - For sample implementations of the
SymbolProcessor
interface, see the following files in the sample project.src/main/kotlin/BuilderProcessor.kt
src/main/kotlin/TestProcessor.kt
- After writing your own processor, register your processor to the package by including
the fully-qualified name of that processor in
resources/META-INF/services/com.google.devtools.ksp.processing.SymbolProcessor
.
- Your main logic should be in the
-
Create another module that contains a workload where you want to try out your processor.
-
In the project's
setting.gradle.kts
, overrideresolutionStrategy
for the KSP plugin. This is necessary because KSP is still in preview and there is no plugin marker published yet.pluginManagement { resolutionStrategy { eachPlugin { when (requested.id.id) { "symbol-processing" -> useModule("com.google.devtools.ksp:symbol-processing:${requested.version}") } } } repositories { gradlePluginPortal() google() } }
-
In the new module's
build.gradle.kts
, do the following:- Apply the
symbol-processing
plugin with the specified version. - Add
ksp(<your processor>)
to the list of dependencies.
- Apply the
-
Run
./gradlew build
. You can find the generated code underbuild/generated/source/ksp
. -
Here's a
sample build.gradle.kts
to apply the KSP plugin to a workload.plugins { id("symbol-processing") version "1.4.10-dev-experimental-20201120" kotlin("jvm") } version = "1.0-SNAPSHOT" repositories { mavenCentral() google() } dependencies { implementation(kotlin("stdlib-jdk8")) implementation(project(":test-processor")) ksp(project(":test-processor")) }
Processor options in SymbolProcessor.init(options: Map<String, String>, ...)
are specified in gradle build scripts:
ksp {
arg("option1", "value1")
arg("option2", "value2")
...
}
By default, IntelliJ or other IDEs don't know about the generated code and therefore references to those generated symbols will be marked unresolvable. To make, for example, IntelliJ be able to reason about the generated symbols, the following paths need to be marked as generated source root:
build/generated/ksp/src/main/kotlin/
build/generated/ksp/src/main/java/
and perhaps also resource directory if your IDE supports them:
build/generated/ksp/src/main/resources
See Migration from old KSP releases
See CONTRIBUTING.md