Runtime Overview¤
Overview¤
A typical runtime consists of the following parts:
Compiled¤
The Compiled
class is responsible for initializing and managing a device.
Compiled
¤
Compiled(
device: str,
allocator: Allocator,
renderer: Optional[Renderer],
compiler: Optional[Compiler],
runtime,
graph=None,
)
Methods:
-
synchronize
–Synchronize all pending operations on the device.
synchronize
¤
synchronize()
Synchronize all pending operations on the device.
This method ensures that all previously queued operations on the device have been completed before proceeding.
Allocator¤
The Allocator
class is responsible for managing memory on the device. There is also a version called the LRUAllocator
, which caches allocated buffers to optimize performance.
Allocator
¤
Methods:
LRUAllocator
¤
LRUAllocator()
Bases: Allocator
The LRU Allocator is responsible for caching buffers. It ensures that buffers are not freed until it is absolutely necessary, optimizing performance.
Methods:
-
alloc
– -
free
– -
free_cache
–
Attributes:
Program¤
The Program
class is created for each loaded program. It is responsible for compiling and executing the program on the device. As an example, here is a ClangProgram
implementation which loads program and runs it.
ClangProgram
¤
Methods:
-
__call__
–
Attributes:
-
fxn
–
Source code in tinygrad/runtime/ops_clang.py
20 21 22 23 24 25 26 |
|
Compiler¤
The Compiler
class compiles the output from the Renderer
and produces it in a device-specific format.
Compiler
¤
Methods:
Attributes:
-
cachekey
–
Source code in tinygrad/device.py
183 |
|
compile
¤
Source code in tinygrad/device.py
184 |
|
compile_cached
¤
Source code in tinygrad/device.py
185 186 187 188 189 190 |
|