Skip to content

Creation

Creation (basic)¤

empty staticmethod ¤

empty(*shape, **kwargs)

Creates an empty tensor with the given shape.

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

t = Tensor.empty(2, 3)
print(t.shape)
(2, 3)
Source code in tinygrad/tensor.py
394
395
396
397
398
399
400
401
402
403
404
405
406
407
@staticmethod
def empty(*shape, **kwargs):
  """
  Creates an empty tensor with the given shape.

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  t = Tensor.empty(2, 3)
  print(t.shape)
  ```
  """
  return Tensor._metaop(MetaOps.EMPTY, argfix(*shape), **kwargs)

zeros staticmethod ¤

zeros(*shape, **kwargs) -> Tensor

Creates a tensor with the given shape, filled with zeros.

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

print(Tensor.zeros(2, 3).numpy())
[[0. 0. 0.]
 [0. 0. 0.]]
print(Tensor.zeros(2, 3, dtype=dtypes.int32).numpy())
[[0 0 0]
 [0 0 0]]

Source code in tinygrad/tensor.py
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
@staticmethod
def zeros(*shape, **kwargs) -> Tensor:
  """
  Creates a tensor with the given shape, filled with zeros.

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  print(Tensor.zeros(2, 3).numpy())
  ```
  ```python exec="true" source="above" session="tensor" result="python"
  print(Tensor.zeros(2, 3, dtype=dtypes.int32).numpy())
  ```
  """
  return Tensor.full(argfix(*shape), 0.0, **kwargs)

ones staticmethod ¤

ones(*shape, **kwargs) -> Tensor

Creates a tensor with the given shape, filled with ones.

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

print(Tensor.ones(2, 3).numpy())
[[1. 1. 1.]
 [1. 1. 1.]]
print(Tensor.ones(2, 3, dtype=dtypes.int32).numpy())
[[1 1 1]
 [1 1 1]]

Source code in tinygrad/tensor.py
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
@staticmethod
def ones(*shape, **kwargs) -> Tensor:
  """
  Creates a tensor with the given shape, filled with ones.

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  print(Tensor.ones(2, 3).numpy())
  ```
  ```python exec="true" source="above" session="tensor" result="python"
  print(Tensor.ones(2, 3, dtype=dtypes.int32).numpy())
  ```
  """
  return Tensor.full(argfix(*shape), 1.0, **kwargs)

full staticmethod ¤

full(
    shape: Tuple[sint, ...], fill_value: ConstType, **kwargs
) -> Tensor

Creates a tensor with the given shape, filled with the given value.

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

print(Tensor.full((2, 3), 42).numpy())
[[42 42 42]
 [42 42 42]]
print(Tensor.full((2, 3), False).numpy())
[[False False False]
 [False False False]]

Source code in tinygrad/tensor.py
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
@staticmethod
def full(shape:Tuple[sint, ...], fill_value:ConstType, **kwargs) -> Tensor:
  """
  Creates a tensor with the given shape, filled with the given value.

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  print(Tensor.full((2, 3), 42).numpy())
  ```
  ```python exec="true" source="above" session="tensor" result="python"
  print(Tensor.full((2, 3), False).numpy())
  ```
  """
  return Tensor(fill_value, **kwargs).reshape((1, )*len(new_shape := argfix(shape))).expand(new_shape)

arange staticmethod ¤

arange(start, stop=None, step=1, **kwargs) -> Tensor

Returns a 1-D tensor of size ceil((stop - start) / step) with values from [start, stop), with spacing between values given by step.

If stop is not specified, values are generated from [0, start) with the given step.

If stop is specified, values are generated from [start, stop) with the given step.

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

print(Tensor.arange(5).numpy())
[0 1 2 3 4]
print(Tensor.arange(5, 10).numpy())
[5 6 7 8 9]
print(Tensor.arange(5, 10, 2).numpy())
[5 7 9]
print(Tensor.arange(5.5, 10, 2).numpy())
[5.5 7.5 9.5]

Source code in tinygrad/tensor.py
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
@staticmethod
def arange(start, stop=None, step=1, **kwargs) -> Tensor:
  """
  Returns a 1-D tensor of size `ceil((stop - start) / step)` with values from `[start, stop)`, with spacing between values given by `step`.

  If `stop` is not specified, values are generated from `[0, start)` with the given `step`.

  If `stop` is specified, values are generated from `[start, stop)` with the given `step`.

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  print(Tensor.arange(5).numpy())
  ```
  ```python exec="true" source="above" session="tensor" result="python"
  print(Tensor.arange(5, 10).numpy())
  ```
  ```python exec="true" source="above" session="tensor" result="python"
  print(Tensor.arange(5, 10, 2).numpy())
  ```
  ```python exec="true" source="above" session="tensor" result="python"
  print(Tensor.arange(5.5, 10, 2).numpy())
  ```
  """
  if stop is None: stop, start = start, 0
  dtype = kwargs.pop("dtype", dtypes.default_float if any(isinstance(x, float) for x in (start, stop, step)) else dtypes.default_int)
  # NOTE: this matches numpy, torch raises RuntimeError if stop-start and step have different signs
  if (output_len:=ceildiv(stop-start, step)) <= 0: return Tensor([], dtype=dtype, **kwargs)
  return (Tensor.full((output_len,), step, dtype=dtype, **kwargs)._cumsum() + (start - step)).cast(dtype)

eye staticmethod ¤

eye(n: int, m: Optional[int] = None, **kwargs) -> Tensor

Returns a 2-D tensor with n rows and m columns, with ones on the diagonal and zeros elsewhere.

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

print(Tensor.eye(3).numpy())
[[1. 0. 0.]
 [0. 1. 0.]
 [0. 0. 1.]]
print(Tensor.eye(2, 4).numpy())
[[1. 0. 0. 0.]
 [0. 1. 0. 0.]]
Source code in tinygrad/tensor.py
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
@staticmethod
def eye(n:int, m:Optional[int]=None, **kwargs) -> Tensor:
  """
  Returns a 2-D tensor with `n` rows and `m` columns, with ones on the diagonal and zeros elsewhere.

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  print(Tensor.eye(3).numpy())
  ```

  ```python exec="true" source="above" session="tensor" result="python"
  print(Tensor.eye(2, 4).numpy())
  ```
  """
  if n < 0 or (m is not None and m < 0): raise ValueError(f"cannot have negative {n=}, {m=}")
  x = Tensor.ones((n,1),**kwargs).pad((None,(0,n))).flatten().shrink(((0,n*n),)).reshape(n,n)
  return x if m is None else x.pad((None, (0, m-n))) if m > n else x.shrink((None, (0, m)))

full_like ¤

full_like(fill_value: ConstType, **kwargs) -> Tensor

Creates a tensor with the same shape as self, filled with the given value. If dtype is not specified, the dtype of self is used.

You can pass in the device keyword argument to control device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

t = Tensor.ones(2, 3)
print(Tensor.full_like(t, 42).numpy())
[[42. 42. 42.]
 [42. 42. 42.]]
Source code in tinygrad/tensor.py
627
628
629
630
631
632
633
634
635
636
637
638
639
640
def full_like(self, fill_value:ConstType, **kwargs) -> Tensor:
  """
  Creates a tensor with the same shape as `self`, filled with the given value.
  If `dtype` is not specified, the dtype of `self` is used.

  You can pass in the `device` keyword argument to control device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  t = Tensor.ones(2, 3)
  print(Tensor.full_like(t, 42).numpy())
  ```
  """
  return Tensor.full(self.shape, fill_value, dtype=kwargs.pop("dtype", self.dtype), device=kwargs.pop("device", self.device), **kwargs)

zeros_like ¤

zeros_like(**kwargs) -> Tensor

Creates a tensor with the same shape as self, filled with zeros.

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

t = Tensor.ones(2, 3)
print(Tensor.zeros_like(t).numpy())
[[0. 0. 0.]
 [0. 0. 0.]]
Source code in tinygrad/tensor.py
642
643
644
645
646
647
648
649
650
651
652
653
654
def zeros_like(self, **kwargs) -> Tensor:
  """
  Creates a tensor with the same shape as `self`, filled with zeros.

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  t = Tensor.ones(2, 3)
  print(Tensor.zeros_like(t).numpy())
  ```
  """
  return self.full_like(0, **kwargs)

ones_like ¤

ones_like(**kwargs) -> Tensor

Creates a tensor with the same shape as self, filled with ones.

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

t = Tensor.zeros(2, 3)
print(Tensor.ones_like(t).numpy())
[[1. 1. 1.]
 [1. 1. 1.]]
Source code in tinygrad/tensor.py
656
657
658
659
660
661
662
663
664
665
666
667
668
def ones_like(self, **kwargs) -> Tensor:
  """
  Creates a tensor with the same shape as `self`, filled with ones.

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  t = Tensor.zeros(2, 3)
  print(Tensor.ones_like(t).numpy())
  ```
  """
  return self.full_like(1, **kwargs)

Creation (external)¤

from_blob staticmethod ¤

from_blob(
    ptr: int, shape: Tuple[int, ...], **kwargs
) -> Tensor

Exposes the pointer as a Tensor without taking ownership of the original data. The pointer must remain valid for the entire lifetime of the created Tensor.

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

Source code in tinygrad/tensor.py
409
410
411
412
413
414
415
416
417
418
419
420
421
422
@staticmethod
def from_blob(ptr:int, shape:Tuple[int, ...], **kwargs) -> Tensor:
  """
  Exposes the pointer as a Tensor without taking ownership of the original data.
  The pointer must remain valid for the entire lifetime of the created Tensor.

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.
  """

  r = Tensor._metaop(MetaOps.EMPTY, shape, **kwargs)
  r.lazydata.buffer.allocate(external_ptr=ptr)
  del r.lazydata.srcs # fake realize
  return r

from_url staticmethod ¤

from_url(
    url: str, gunzip: bool = False, **kwargs
) -> Tensor

Create a Tensor from a URL.

This is the preferred way to access Internet resources. It currently returns a DISK Tensor, but in the future it may return an HTTP Tensor. This also will soon become lazy (when possible) and not print progress without DEBUG.

THe gunzip flag will gzip extract the resource and return an extracted Tensor.

Source code in tinygrad/tensor.py
424
425
426
427
428
429
430
431
432
433
434
435
@staticmethod
def from_url(url:str, gunzip:bool=False, **kwargs) -> Tensor:
  """
  Create a Tensor from a URL.

  This is the preferred way to access Internet resources.
  It currently returns a DISK Tensor, but in the future it may return an HTTP Tensor.
  This also will soon become lazy (when possible) and not print progress without DEBUG.

  THe `gunzip` flag will gzip extract the resource and return an extracted Tensor.
  """
  return Tensor(fetch(url, gunzip=gunzip), **kwargs)

Creation (random)¤

manual_seed staticmethod ¤

manual_seed(seed=0)

Sets the seed for random operations.

Tensor.manual_seed(42)
print(Tensor.rand(5).numpy())
print(Tensor.rand(5).numpy())
[0.997  0.5899 0.2225 0.7551 0.9057]
[0.6162 0.6213 0.9791 0.7851 0.4178]
Tensor.manual_seed(42)  # reset to the same seed
print(Tensor.rand(5).numpy())
print(Tensor.rand(5).numpy())
[0.997  0.5899 0.2225 0.7551 0.9057]
[0.6162 0.6213 0.9791 0.7851 0.4178]

Source code in tinygrad/tensor.py
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
@staticmethod
def manual_seed(seed=0):
  """
  Sets the seed for random operations.

  ```python exec="true" source="above" session="tensor" result="python"
  Tensor.manual_seed(42)
  print(Tensor.rand(5).numpy())
  print(Tensor.rand(5).numpy())
  ```
  ```python exec="true" source="above" session="tensor" result="python"
  Tensor.manual_seed(42)  # reset to the same seed
  print(Tensor.rand(5).numpy())
  print(Tensor.rand(5).numpy())
  ```
  """
  Tensor._seed, Tensor._device_seeds, Tensor._device_rng_counters = seed, {}, {}

rand staticmethod ¤

rand(
    *shape,
    device: Optional[str] = None,
    dtype: Optional[DTypeLike] = None,
    contiguous: bool = True,
    **kwargs
) -> Tensor

Creates a tensor with the given shape, filled with random values from a uniform distribution over the interval [0, 1).

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

Tensor.manual_seed(42)
t = Tensor.rand(2, 3)
print(t.numpy())
[[0.997  0.5899 0.2225]
 [0.7551 0.9057 0.8649]]
Source code in tinygrad/tensor.py
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
@staticmethod
def rand(*shape, device:Optional[str]=None, dtype:Optional[DTypeLike]=None, contiguous:bool=True, **kwargs) -> Tensor:
  """
  Creates a tensor with the given shape, filled with random values from a uniform distribution over the interval `[0, 1)`.

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  Tensor.manual_seed(42)
  t = Tensor.rand(2, 3)
  print(t.numpy())
  ```
  """
  if not dtypes.is_float(dtype := to_dtype(dtype or dtypes.default_float)): raise ValueError(f"rand only supports float dtypes, got {dtype}")
  if not all_int(shape:=argfix(*shape)) or not all(s >= 0 for s in shape): raise ValueError(f"invalid input {shape=}")
  if device is not None and not isinstance(device, str): raise ValueError(f"rand only supports single device, got {device=}")
  _device = device = Device.canonicalize(device)

  # when using MOCKGPU and NV generate rand on CLANG
  if getenv("MOCKGPU") and device.startswith("NV"): device = "CLANG"

  # generate per device seeds and rng counter if we haven't seen this device yet
  if device not in Tensor._device_seeds:
    Tensor._device_seeds[device] = Tensor([((Tensor._seed & 0xffffffff) << 32) \
      | int.from_bytes(hashlib.sha256(len(Tensor._device_seeds).to_bytes(4, "big")).digest(), "big") & 0xffffffff],
                                          device=device, dtype=dtypes.uint64, requires_grad=False)
    Tensor._device_rng_counters[device] = Tensor([0], device=device, dtype=dtypes.uint32, requires_grad=False)
    had_counter = False
  else: had_counter = True

  # if shape has 0, return zero tensor
  if (num := ceildiv(((num_ := prod(shape)) * dtype.itemsize), 4)) == 0: return Tensor.zeros(shape, device=_device, dtype=dtype, **kwargs)

  # increment rng counter for devices
  if had_counter: Tensor._device_rng_counters[device].assign(Tensor._device_rng_counters[device] + num).contiguous()

  # threefry random bits
  counts0 = (Tensor.arange(ceildiv(num, 2), device=device, dtype=dtypes.uint32, requires_grad=False)+Tensor._device_rng_counters[device])
  counts1 = counts0 + ceildiv(num, 2)
  bits = Tensor._threefry_random_bits(Tensor._device_seeds[device], counts0, counts1)[:num]

  # bitcast to uint with same number of bits
  _, nmant = dtypes.finfo(dtype)
  uint_dtype = {1: dtypes.uint8, 2: dtypes.uint16, 4: dtypes.uint32, 8: dtypes.uint64}[dtype.itemsize]
  bits = bits.bitcast(uint_dtype)
  # only randomize the mantissa bits and set the exponent to 1
  one = Tensor.ones_like(bits, device=bits.device, dtype=dtype).bitcast(uint_dtype)
  bits = bits.rshift((dtype.itemsize * 8) - nmant).bitwise_or(one)
  # bitcast back to the original dtype and reshape
  out = bits.bitcast(dtype)[:num_].sub(1).reshape(shape)

  # move back to the original device if we were using MOCKGPU
  if getenv("MOCKGPU") and _device: out = out.to(_device)

  out.requires_grad = kwargs.get("requires_grad")
  return out.contiguous() if contiguous else out

randn staticmethod ¤

randn(
    *shape, dtype: Optional[DTypeLike] = None, **kwargs
) -> Tensor

Creates a tensor with the given shape, filled with random values from a normal distribution with mean 0 and standard deviation 1. If dtype is not specified, the default type is used.

You can pass in the device keyword argument to control device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

Tensor.manual_seed(42)
print(Tensor.randn(2, 3).numpy())
[[ 0.9779  0.4678  0.5526]
 [-0.3288 -0.8555  0.2753]]
Source code in tinygrad/tensor.py
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
@staticmethod
def randn(*shape, dtype:Optional[DTypeLike]=None, **kwargs) -> Tensor:
  """
  Creates a tensor with the given shape, filled with random values from a normal distribution with mean `0` and standard deviation `1`.
  If `dtype` is not specified, the default type is used.

  You can pass in the `device` keyword argument to control device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  Tensor.manual_seed(42)
  print(Tensor.randn(2, 3).numpy())
  ```
  """
  # https://en.wikipedia.org/wiki/Box%E2%80%93Muller_transform
  src = Tensor.rand((2, *argfix(*shape)), **{**kwargs, "dtype": dtypes.float32})
  return src[0].mul(2*math.pi).cos().mul((1 - src[1]).log().mul(-2).sqrt()).cast(dtype or dtypes.default_float)

randint staticmethod ¤

randint(*shape, low=0, high=10, **kwargs) -> Tensor

Creates a tensor with the given shape, filled with random integer values generated uniformly from the interval [low, high). If dtype is not specified, the default type is used.

You can pass in the device keyword argument to control device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

Tensor.manual_seed(42)
print(Tensor.randint(2, 3, low=5, high=10).numpy())
[[9 7 6]
 [8 9 9]]
Source code in tinygrad/tensor.py
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
@staticmethod
def randint(*shape, low=0, high=10, **kwargs) -> Tensor:
  """
  Creates a tensor with the given shape, filled with random integer values generated uniformly from the interval `[low, high)`.
  If `dtype` is not specified, the default type is used.

  You can pass in the `device` keyword argument to control device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  Tensor.manual_seed(42)
  print(Tensor.randint(2, 3, low=5, high=10).numpy())
  ```
  """
  if not isinstance(low, int) or not isinstance(high, int): raise TypeError(f"{low=} and {high=} must be integers")
  dtype = kwargs.pop("dtype", dtypes.int32)
  if not dtypes.is_int(dtype): raise TypeError(f"{dtype=} must be int")
  return Tensor.uniform(*shape, low=low, high=high, dtype=dtype, **kwargs)

normal staticmethod ¤

normal(*shape, mean=0.0, std=1.0, **kwargs) -> Tensor

Creates a tensor with the given shape, filled with random values from a normal distribution with the given mean and standard deviation std.

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

Tensor.manual_seed(42)
print(Tensor.normal(2, 3, mean=10, std=2).numpy())
[[11.9557 10.9356 11.1053]
 [ 9.3423  8.289  10.5505]]
Source code in tinygrad/tensor.py
733
734
735
736
737
738
739
740
741
742
743
744
745
746
@staticmethod
def normal(*shape, mean=0.0, std=1.0, **kwargs) -> Tensor:
  """
  Creates a tensor with the given shape, filled with random values from a normal distribution with the given `mean` and standard deviation `std`.

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  Tensor.manual_seed(42)
  print(Tensor.normal(2, 3, mean=10, std=2).numpy())
  ```
  """
  return (std * Tensor.randn(*shape, **kwargs)) + mean

uniform staticmethod ¤

uniform(*shape, low=0.0, high=1.0, **kwargs) -> Tensor

Creates a tensor with the given shape, filled with random values from a uniform distribution over the interval [low, high).

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

Tensor.manual_seed(42)
print(Tensor.uniform(2, 3, low=2, high=10).numpy())
[[9.9763 6.7193 3.7804]
 [8.0404 9.2452 8.9191]]
Source code in tinygrad/tensor.py
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
@staticmethod
def uniform(*shape, low=0.0, high=1.0, **kwargs) -> Tensor:
  """
  Creates a tensor with the given shape, filled with random values from a uniform distribution over the interval `[low, high)`.

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  Tensor.manual_seed(42)
  print(Tensor.uniform(2, 3, low=2, high=10).numpy())
  ```
  """
  dtype = kwargs.pop("dtype", dtypes.default_float)
  return ((high-low) * Tensor.rand(*shape, **kwargs)).cast(dtype) + low

scaled_uniform staticmethod ¤

scaled_uniform(*shape, **kwargs) -> Tensor

Creates a tensor with the given shape, filled with random values from a uniform distribution over the interval [-prod(shape)**-0.5, prod(shape)**-0.5).

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

Tensor.manual_seed(42)
print(Tensor.scaled_uniform(2, 3).numpy())
[[ 0.4058  0.0734 -0.2265]
 [ 0.2082  0.3312  0.2979]]
Source code in tinygrad/tensor.py
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
@staticmethod
def scaled_uniform(*shape, **kwargs) -> Tensor:
  """
  Creates a tensor with the given shape, filled with random values from a uniform distribution
  over the interval `[-prod(shape)**-0.5, prod(shape)**-0.5)`.

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  Tensor.manual_seed(42)
  print(Tensor.scaled_uniform(2, 3).numpy())
  ```
  """
  return Tensor.uniform(*shape, low=-1.0, high=1.0, **kwargs).mul(prod(argfix(*shape))**-0.5)

glorot_uniform staticmethod ¤

glorot_uniform(*shape, **kwargs) -> Tensor

https://www.tensorflow.org/api_docs/python/tf/keras/initializers/GlorotUniform

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

Tensor.manual_seed(42)
print(Tensor.glorot_uniform(2, 3).numpy())
[[ 1.0889  0.197  -0.6079]
 [ 0.5588  0.8887  0.7994]]
Source code in tinygrad/tensor.py
781
782
783
784
785
786
787
788
789
790
791
792
793
794
@staticmethod
def glorot_uniform(*shape, **kwargs) -> Tensor:
  """
  <https://www.tensorflow.org/api_docs/python/tf/keras/initializers/GlorotUniform>

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  Tensor.manual_seed(42)
  print(Tensor.glorot_uniform(2, 3).numpy())
  ```
  """
  return Tensor.uniform(*shape, low=-1.0, high=1.0, **kwargs).mul((6/(argfix(*shape)[0]+prod(argfix(*shape)[1:])))**0.5)

kaiming_uniform staticmethod ¤

kaiming_uniform(
    *shape, a: float = 0.01, **kwargs
) -> Tensor

https://pytorch.org/docs/stable/_modules/torch/nn/init.html#kaiming_uniform_

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

Tensor.manual_seed(42)
print(Tensor.kaiming_uniform(2, 3).numpy())
[[ 1.4058  0.2543 -0.7847]
 [ 0.7214  1.1473  1.032 ]]
Source code in tinygrad/tensor.py
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
@staticmethod
def kaiming_uniform(*shape, a:float = 0.01, **kwargs) -> Tensor:
  """
  <https://pytorch.org/docs/stable/_modules/torch/nn/init.html#kaiming_uniform_>

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  Tensor.manual_seed(42)
  print(Tensor.kaiming_uniform(2, 3).numpy())
  ```
  """
  bound = math.sqrt(3.0) * math.sqrt(2.0 / (1 + a ** 2)) / math.sqrt(prod(argfix(*shape)[1:]))
  return Tensor.uniform(*shape, low=-bound, high=bound, **kwargs)

kaiming_normal staticmethod ¤

kaiming_normal(*shape, a: float = 0.01, **kwargs) -> Tensor

https://pytorch.org/docs/stable/_modules/torch/nn/init.html#kaiming_normal_

You can pass in dtype and device keyword arguments to control the data type and device of the tensor. Additionally, all other keyword arguments are passed to the constructor of the tensor.

Tensor.manual_seed(42)
print(Tensor.kaiming_normal(2, 3).numpy())
[[ 0.7984  0.3819  0.4512]
 [-0.2685 -0.6985  0.2247]]
Source code in tinygrad/tensor.py
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
@staticmethod
def kaiming_normal(*shape, a:float = 0.01, **kwargs) -> Tensor:
  """
  <https://pytorch.org/docs/stable/_modules/torch/nn/init.html#kaiming_normal_>

  You can pass in `dtype` and `device` keyword arguments to control the data type and device of the tensor.
  Additionally, all other keyword arguments are passed to the constructor of the tensor.

  ```python exec="true" source="above" session="tensor" result="python"
  Tensor.manual_seed(42)
  print(Tensor.kaiming_normal(2, 3).numpy())
  ```
  """
  std = math.sqrt(2.0 / (1 + a ** 2)) / math.sqrt(prod(argfix(*shape)[1:]))
  return Tensor.normal(*shape, mean=0.0, std=std, **kwargs)