Runtime Overview¤
Overview¤
A typical runtime consists of the following parts:
Compiled¤
The Compiled class is responsible for initializing and managing a device.
Compiled
¤
Compiled(
device: str,
allocator: Allocator,
runtime: type[Program[Self]] | None,
arch=None,
)
Allocator¤
The Allocator class manages memory on the device and caches allocated buffers for reuse.
Allocator
¤
Allocator(dev: DeviceType)
Bases: Generic[DeviceType]
Methods:
-
_alloc– -
_copyin– -
_copyout– -
_free– -
_map– -
_offset– -
_unmap– -
alloc– -
do_free– -
free– -
free_cache– -
map–
Attributes:
Program¤
The Program class is created for each loaded program. It is responsible for executing the program on the device. As an example, here is a CPUProgram implementation which loads program and runs it.
CPUProgram
¤
CPUProgram(
dev: CPUDevice,
obj: TinyELF,
links: dict[str, CPUProgram],
)
Bases: Program['CPUDevice']
Methods:
Attributes:
Source code in tinygrad/runtime/ops_cpu.py
22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | |
fxn
instance-attribute
¤
fxn: Any = dev.remote.rpc(
RemoteCmd.LOAD_PROG,
len(obj.lib),
len(link_ids),
payload=obj.lib + link_ids,
)[0]
packer
instance-attribute
¤
packer = struct.Struct(
"".join(
unwrap(dt.fmt)
for _, dt in TinyELF.iter_sig(self.signature)
)
)
__call__
¤
__call__(
*args: int,
global_size: tuple[int, int, int] = (1, 1, 1),
local_size: tuple[int, int, int] = (1, 1, 1),
wait: bool = False,
timeout: int | None = None
) -> float | None
Source code in tinygrad/runtime/ops_cpu.py
48 49 50 51 52 53 54 55 56 57 58 | |
_load
¤
_load(lib, base=0)
Source code in tinygrad/runtime/ops_cpu.py
19 20 | |
Compiler¤
The Compiler class compiles the output from the Renderer and produces it in a device-specific format.
Compiler
¤
Compiler(cachekey: str | None = None)
Methods:
-
compile– -
compile_cached– -
compile_server– -
disassemble– -
server–
Attributes:
-
cachekey–
Source code in tinygrad/renderer/__init__.py
13 | |
compile
¤
compile(src: str) -> bytes
Source code in tinygrad/renderer/__init__.py
14 | |
compile_cached
¤
compile_cached(src: str) -> bytes
Source code in tinygrad/renderer/__init__.py
15 16 17 18 19 20 | |
compile_server
¤
compile_server(src: str, proc: Popen) -> bytes
Source code in tinygrad/renderer/__init__.py
25 26 27 28 | |
disassemble
¤
disassemble(lib: bytes)
Source code in tinygrad/renderer/__init__.py
21 | |
server
¤
server(cmd: str, arch: str, *args) -> Popen
Source code in tinygrad/renderer/__init__.py
22 23 24 | |