Elementwise
Elementwise ops operate on a per element basis. They don't change the shape of the tensor.
Unary Ops (math)¤
logical_not
¤
logical_not() -> Tensor
Computes the logical NOT of the tensor element-wise.
print(Tensor([False, True]).logical_not().numpy())
[ True False]
Source code in tinygrad/tensor.py
3004 3005 3006 3007 3008 3009 3010 3011 3012 |
|
neg
¤
neg() -> Tensor
Negates the tensor element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).neg().numpy())
[ 3. 2. 1. -0. -1. -2. -3.]
Source code in tinygrad/tensor.py
3014 3015 3016 3017 3018 3019 3020 3021 3022 |
|
log
¤
log() -> Tensor
Computes the natural logarithm element-wise.
See: https://en.wikipedia.org/wiki/Logarithm
print(Tensor([1., 2., 4., 8.]).log().numpy())
[0. 0.6931 1.3863 2.0794]
Source code in tinygrad/tensor.py
3045 3046 3047 3048 3049 3050 3051 3052 3053 3054 3055 |
|
log2
¤
log2() -> Tensor
Computes the base-2 logarithm element-wise.
See: https://en.wikipedia.org/wiki/Logarithm
print(Tensor([1., 2., 4., 8.]).log2().numpy())
[0. 1. 2. 3.]
Source code in tinygrad/tensor.py
3057 3058 3059 3060 3061 3062 3063 3064 3065 3066 3067 |
|
exp
¤
exp() -> Tensor
Computes the exponential function element-wise.
See: https://en.wikipedia.org/wiki/Exponential_function
print(Tensor([0., 1., 2., 3.]).exp().numpy())
[ 1. 2.7183 7.3891 20.0855]
Source code in tinygrad/tensor.py
3069 3070 3071 3072 3073 3074 3075 3076 3077 3078 3079 3080 3081 3082 |
|
exp2
¤
exp2() -> Tensor
Computes the base-2 exponential function element-wise.
See: https://en.wikipedia.org/wiki/Exponential_function
print(Tensor([0., 1., 2., 3.]).exp2().numpy())
[1. 2. 4. 8.]
Source code in tinygrad/tensor.py
3084 3085 3086 3087 3088 3089 3090 3091 3092 3093 3094 |
|
sqrt
¤
sqrt() -> Tensor
Computes the square root of the tensor element-wise.
print(Tensor([1., 2., 3., 4.]).sqrt().numpy())
[1. 1.4142 1.7321 2. ]
Source code in tinygrad/tensor.py
3144 3145 3146 3147 3148 3149 3150 3151 3152 |
|
rsqrt
¤
rsqrt() -> Tensor
Computes the reciprocal of the square root of the tensor element-wise.
print(Tensor([1., 2., 3., 4.]).rsqrt().numpy())
[1. 0.7071 0.5774 0.5 ]
Source code in tinygrad/tensor.py
3154 3155 3156 3157 3158 3159 3160 3161 3162 |
|
sin
¤
sin() -> Tensor
Computes the sine of the tensor element-wise.
print(Tensor([0., math.pi/2, math.pi, 3*math.pi/2, 2*math.pi]).sin().numpy())
[ 0. 1. -0. -1. 0.]
Source code in tinygrad/tensor.py
3164 3165 3166 3167 3168 3169 3170 3171 3172 |
|
cos
¤
cos() -> Tensor
Computes the cosine of the tensor element-wise.
print(Tensor([0., math.pi/2, math.pi, 3*math.pi/2, 2*math.pi]).cos().numpy())
[ 1.0000e+00 0.0000e+00 -1.0000e+00 -2.3842e-07 1.0000e+00]
Source code in tinygrad/tensor.py
3174 3175 3176 3177 3178 3179 3180 3181 3182 3183 |
|
tan
¤
tan() -> Tensor
Computes the tangent of the tensor element-wise.
print(Tensor([0., math.pi/4, math.pi/2, 3*math.pi/4, math.pi]).tan().numpy())
[ 0. 1. inf -1. 0.]
Source code in tinygrad/tensor.py
3185 3186 3187 3188 3189 3190 3191 3192 3193 |
|
asin
¤
asin() -> Tensor
Computes the inverse sine (arcsine) of the tensor element-wise.
print(Tensor([-0.9, -0.6, -0.3, 0., 0.3, 0.6, 0.9]).asin().numpy())
[-1.1198 -0.6435 -0.3047 0. 0.3047 0.6435 1.1198]
Source code in tinygrad/tensor.py
3195 3196 3197 3198 3199 3200 3201 3202 3203 3204 3205 3206 |
|
acos
¤
acos() -> Tensor
Computes the inverse cosine (arccosine) of the tensor element-wise.
print(Tensor([-0.9, -0.6, -0.3, 0., 0.3, 0.6, 0.9]).acos().numpy())
[2.6906 2.2143 1.8755 1.5708 1.2661 0.9273 0.451 ]
Source code in tinygrad/tensor.py
3208 3209 3210 3211 3212 3213 3214 3215 3216 |
|
atan
¤
atan() -> Tensor
Computes the inverse tangent (arctan) of the tensor element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).atan().numpy())
[-1.249 -1.1071 -0.7854 0. 0.7854 1.1071 1.249 ]
Source code in tinygrad/tensor.py
3218 3219 3220 3221 3222 3223 3224 3225 3226 |
|
trunc
¤
trunc() -> Tensor
Truncates the tensor element-wise.
print(Tensor([-3.5, -2.5, -1.5, -0.5, 0.5, 1.5, 2.5, 3.5]).trunc().numpy())
[-3. -2. -1. -0. 0. 1. 2. 3.]
Source code in tinygrad/tensor.py
3230 3231 3232 3233 3234 3235 3236 3237 3238 |
|
ceil
¤
ceil() -> Tensor
Rounds the tensor element-wise towards positive infinity.
print(Tensor([-3.5, -2.5, -1.5, -0.5, 0.5, 1.5, 2.5, 3.5]).ceil().numpy())
[-3. -2. -1. -0. 1. 2. 3. 4.]
Source code in tinygrad/tensor.py
3240 3241 3242 3243 3244 3245 3246 3247 3248 |
|
floor
¤
floor() -> Tensor
Rounds the tensor element-wise towards negative infinity.
print(Tensor([-3.5, -2.5, -1.5, -0.5, 0.5, 1.5, 2.5, 3.5]).floor().numpy())
[-4. -3. -2. -1. 0. 1. 2. 3.]
Source code in tinygrad/tensor.py
3250 3251 3252 3253 3254 3255 3256 3257 3258 |
|
round
¤
round() -> Tensor
Rounds the tensor element-wise with rounding half to even.
print(Tensor([-3.5, -2.5, -1.5, -0.5, 0.5, 1.5, 2.5, 3.5]).round().numpy())
[-4. -2. -2. 0. 0. 2. 2. 4.]
Source code in tinygrad/tensor.py
3260 3261 3262 3263 3264 3265 3266 3267 3268 |
|
isinf
¤
Checks the tensor element-wise to return True where the element is infinity, otherwise returns False
print(Tensor([1, float('inf'), 2, float('-inf'), float('nan')]).isinf().numpy())
[False True False True False]
Source code in tinygrad/tensor.py
3270 3271 3272 3273 3274 3275 3276 3277 3278 |
|
isnan
¤
isnan() -> Tensor
Checks the tensor element-wise to return True where the element is NaN, otherwise returns False
print(Tensor([1, float('inf'), 2, float('-inf'), float('nan')]).isnan().numpy())
[False False False False True]
Source code in tinygrad/tensor.py
3280 3281 3282 3283 3284 3285 3286 3287 3288 |
|
isfinite
¤
isfinite() -> Tensor
Checks the tensor element-wise to return True where the element is finite, otherwise returns False
print(Tensor([1, float('inf'), 2, float('-inf'), float('nan')]).isfinite().numpy())
[ True False True False False]
Source code in tinygrad/tensor.py
3290 3291 3292 3293 3294 3295 3296 3297 3298 |
|
lerp
¤
Linearly interpolates between self
and end
by weight
.
print(Tensor([1., 2., 3.]).lerp(Tensor([4., 5., 6.]), 0.5).numpy())
[2.5 3.5 4.5]
Source code in tinygrad/tensor.py
3300 3301 3302 3303 3304 3305 3306 3307 3308 3309 3310 3311 |
|
square
¤
square() -> Tensor
Squares the tensor element-wise.
Equivalent to self*self
.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).square().numpy())
[9. 4. 1. 0. 1. 4. 9.]
Source code in tinygrad/tensor.py
3313 3314 3315 3316 3317 3318 3319 3320 3321 3322 |
|
clamp
¤
clamp(min_=None, max_=None) -> Tensor
Clips (clamps) the values in the tensor between min_
and max_
element-wise.
If min_
is None
, there is no lower bound. If max_
is None, there is no upper bound.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).clip(-1, 1).numpy())
[-1. -1. -1. 0. 1. 1. 1.]
Source code in tinygrad/tensor.py
3324 3325 3326 3327 3328 3329 3330 3331 3332 3333 3334 3335 |
|
clip
¤
clip(min_=None, max_=None) -> Tensor
Alias for Tensor.clamp
.
Source code in tinygrad/tensor.py
3337 3338 3339 3340 3341 |
|
sign
¤
sign() -> Tensor
Returns the sign of the tensor element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).sign().numpy())
[-1. -1. -1. 0. 1. 1. 1.]
Source code in tinygrad/tensor.py
3343 3344 3345 3346 3347 3348 3349 3350 3351 |
|
abs
¤
abs() -> Tensor
Computes the absolute value of the tensor element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).abs().numpy())
[3. 2. 1. 0. 1. 2. 3.]
Source code in tinygrad/tensor.py
3353 3354 3355 3356 3357 3358 3359 3360 3361 |
|
reciprocal
¤
reciprocal() -> Tensor
Computes 1/x
element-wise.
print(Tensor([1., 2., 3., 4.]).reciprocal().numpy())
[1. 0.5 0.3333 0.25 ]
Source code in tinygrad/tensor.py
3363 3364 3365 3366 3367 3368 3369 3370 3371 |
|
Unary Ops (activation)¤
relu
¤
relu() -> Tensor
Applies the Rectified Linear Unit (ReLU) function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).relu().numpy())
[0. 0. 0. 0. 1. 2. 3.]
Source code in tinygrad/tensor.py
3096 3097 3098 3099 3100 3101 3102 3103 3104 3105 |
|
sigmoid
¤
sigmoid() -> Tensor
Applies the Sigmoid function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).sigmoid().numpy())
[0.0474 0.1192 0.2689 0.5 0.7311 0.8808 0.9526]
Source code in tinygrad/tensor.py
3107 3108 3109 3110 3111 3112 3113 3114 3115 3116 3117 |
|
logsigmoid
¤
logsigmoid() -> Tensor
Applies the LogSigmoid function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).logsigmoid().numpy())
[-3.0486 -2.1269 -1.3133 -0.6931 -0.3133 -0.1269 -0.0486]
Source code in tinygrad/tensor.py
3119 3120 3121 3122 3123 3124 3125 3126 3127 3128 3129 |
|
hardsigmoid
¤
Applies the Hardsigmoid function element-wise.
NOTE: default alpha
and beta
values are taken from torch
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).hardsigmoid().numpy())
[0. 0.1667 0.3333 0.5 0.6667 0.8333 1. ]
Source code in tinygrad/tensor.py
3131 3132 3133 3134 3135 3136 3137 3138 3139 3140 3141 3142 |
|
elu
¤
elu(alpha=1.0) -> Tensor
Applies the Exponential Linear Unit (ELU) function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).elu().numpy())
[-0.9502 -0.8647 -0.6321 0. 1. 2. 3. ]
Source code in tinygrad/tensor.py
3375 3376 3377 3378 3379 3380 3381 3382 3383 3384 3385 |
|
celu
¤
celu(alpha=1.0) -> Tensor
Applies the Continuously differentiable Exponential Linear Unit (CELU) function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).celu().numpy())
[-0.9502 -0.8647 -0.6321 0. 1. 2. 3. ]
Source code in tinygrad/tensor.py
3387 3388 3389 3390 3391 3392 3393 3394 3395 3396 3397 |
|
selu
¤
selu(alpha=1.67326, gamma=1.0507) -> Tensor
Applies the Scaled Exponential Linear Unit (SELU) function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).selu().numpy())
[-1.6706 -1.5202 -1.1113 0. 1.0507 2.1014 3.1521]
Source code in tinygrad/tensor.py
3399 3400 3401 3402 3403 3404 3405 3406 3407 3408 3409 |
|
swish
¤
swish() -> Tensor
See .silu()
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).swish().numpy())
[-0.1423 -0.2384 -0.2689 0. 0.7311 1.7616 2.8577]
Source code in tinygrad/tensor.py
3411 3412 3413 3414 3415 3416 3417 3418 3419 3420 3421 |
|
silu
¤
silu() -> Tensor
Applies the Sigmoid Linear Unit (SiLU) function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).silu().numpy())
[-0.1423 -0.2384 -0.2689 0. 0.7311 1.7616 2.8577]
Source code in tinygrad/tensor.py
3423 3424 3425 3426 3427 3428 3429 3430 3431 3432 3433 |
|
relu6
¤
relu6() -> Tensor
Applies the ReLU6 function element-wise.
print(Tensor([-9., -6., -3., 0., 3., 6., 9.]).relu6().numpy())
[0. 0. 0. 0. 3. 6. 6.]
Source code in tinygrad/tensor.py
3435 3436 3437 3438 3439 3440 3441 3442 3443 3444 3445 |
|
hardswish
¤
hardswish() -> Tensor
Applies the Hardswish function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).hardswish().numpy())
[-0. -0.3333 -0.3333 0. 0.6667 1.6667 3. ]
Source code in tinygrad/tensor.py
3447 3448 3449 3450 3451 3452 3453 3454 3455 3456 3457 |
|
tanh
¤
tanh() -> Tensor
Applies the Hyperbolic Tangent (tanh) function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).tanh().numpy())
[-0.9951 -0.964 -0.7616 0. 0.7616 0.964 0.9951]
Source code in tinygrad/tensor.py
3459 3460 3461 3462 3463 3464 3465 3466 3467 3468 3469 |
|
sinh
¤
sinh() -> Tensor
Applies the Hyperbolic Sine (sinh) function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).sinh().numpy())
[-10.0179 -3.6269 -1.1752 0. 1.1752 3.6269 10.0179]
Source code in tinygrad/tensor.py
3471 3472 3473 3474 3475 3476 3477 3478 3479 3480 3481 |
|
cosh
¤
cosh() -> Tensor
Applies the Hyperbolic Cosine (cosh) function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).cosh().numpy())
[10.0677 3.7622 1.5431 1. 1.5431 3.7622 10.0677]
Source code in tinygrad/tensor.py
3483 3484 3485 3486 3487 3488 3489 3490 3491 3492 3493 |
|
atanh
¤
atanh() -> Tensor
Applies the Inverse Hyperbolic Tangent (atanh) function element-wise.
print(Tensor([-0.9, -0.6, -0.3, 0., 0.3, 0.6, 0.9]).atanh().numpy())
[-1.4722 -0.6931 -0.3095 0. 0.3095 0.6931 1.4722]
Source code in tinygrad/tensor.py
3495 3496 3497 3498 3499 3500 3501 3502 3503 3504 3505 |
|
asinh
¤
asinh() -> Tensor
Applies the Inverse Hyperbolic Sine (asinh) function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).asinh().numpy())
[-1.8184 -1.4436 -0.8814 0. 0.8814 1.4436 1.8184]
Source code in tinygrad/tensor.py
3507 3508 3509 3510 3511 3512 3513 3514 3515 3516 3517 |
|
acosh
¤
acosh() -> Tensor
Applies the Inverse Hyperbolic Cosine (acosh) function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).acosh().numpy())
[ nan nan nan nan 0. 1.317 1.7627]
Source code in tinygrad/tensor.py
3519 3520 3521 3522 3523 3524 3525 3526 3527 3528 3529 |
|
hardtanh
¤
hardtanh(min_val=-1, max_val=1) -> Tensor
Applies the Hardtanh function element-wise.
print(Tensor([-1.5, -1.0, -0.5, 0., 0.5, 1.0, 1.5]).hardtanh().numpy())
[-1. -1. -0.5 0. 0.5 1. 1. ]
Source code in tinygrad/tensor.py
3531 3532 3533 3534 3535 3536 3537 3538 3539 |
|
erf
¤
erf() -> Tensor
Applies error function element-wise.
- Described: https://en.wikipedia.org/wiki/Error_function
print(Tensor([-1.5, -1.0, -0.5, 0., 0.5, 1.0, 1.5]).erf().numpy())
[-0.9661 -0.8427 -0.5205 0. 0.5205 0.8427 0.9661]
Source code in tinygrad/tensor.py
3541 3542 3543 3544 3545 3546 3547 3548 3549 3550 3551 3552 3553 |
|
gelu
¤
gelu() -> Tensor
Applies the Gaussian Error Linear Unit (GELU) function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).gelu().numpy())
[-0.0036 -0.0454 -0.1588 0. 0.8412 1.9546 2.9964]
Source code in tinygrad/tensor.py
3555 3556 3557 3558 3559 3560 3561 3562 3563 3564 3565 |
|
quick_gelu
¤
quick_gelu() -> Tensor
Applies the Sigmoid GELU approximation element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).quick_gelu().numpy())
[-0.0181 -0.0643 -0.1542 0. 0.8458 1.9357 2.9819]
Source code in tinygrad/tensor.py
3567 3568 3569 3570 3571 3572 3573 3574 3575 |
|
leaky_relu
¤
leaky_relu(neg_slope=0.01) -> Tensor
Applies the Leaky ReLU function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).leaky_relu().numpy())
[-0.03 -0.02 -0.01 0. 1. 2. 3. ]
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).leaky_relu(neg_slope=0.42).numpy())
[-1.26 -0.84 -0.42 0. 1. 2. 3. ]
Source code in tinygrad/tensor.py
3577 3578 3579 3580 3581 3582 3583 3584 3585 3586 3587 3588 |
|
mish
¤
mish() -> Tensor
Applies the Mish function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).mish().numpy())
[-0.1456 -0.2525 -0.3034 0. 0.8651 1.944 2.9865]
Source code in tinygrad/tensor.py
3590 3591 3592 3593 3594 3595 3596 3597 3598 3599 3600 |
|
softplus
¤
softplus(beta=1.0) -> Tensor
Applies the Softplus function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).softplus().numpy())
[0.0486 0.1269 0.3133 0.6931 1.3133 2.1269 3.0486]
Source code in tinygrad/tensor.py
3602 3603 3604 3605 3606 3607 3608 3609 3610 |
|
softsign
¤
softsign() -> Tensor
Applies the Softsign function element-wise.
print(Tensor([-3., -2., -1., 0., 1., 2., 3.]).softsign().numpy())
[-0.75 -0.6667 -0.5 0. 0.5 0.6667 0.75 ]
Source code in tinygrad/tensor.py
3612 3613 3614 3615 3616 3617 3618 3619 3620 |
|
Elementwise Ops (broadcasted)¤
add
¤
Adds self
and x
.
Equivalent to self + x
.
Supports broadcasting to a common shape, type promotion, and integer, float, boolean inputs.
Tensor.manual_seed(42)
t = Tensor.randn(4)
print(t.numpy())
[-0.5144 1.085 0.9089 -0.0841]
print(t.add(20).numpy())
[19.4856 21.085 20.9089 19.9159]
print(t.add(Tensor([[2.0], [3.5]])).numpy())
[[1.4856 3.085 2.9089 1.9159]
[2.9856 4.585 4.4089 3.4159]]
Source code in tinygrad/uop/mathtraits.py
23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
|
sub
¤
Subtracts x
from self
.
Equivalent to self - x
.
Supports broadcasting to a common shape, type promotion, and integer, float, boolean inputs.
Tensor.manual_seed(42)
t = Tensor.randn(4)
print(t.numpy())
[-0.5144 1.085 0.9089 -0.0841]
print(t.sub(20).numpy())
[-20.5144 -18.915 -19.0911 -20.0841]
print(t.sub(Tensor([[2.0], [3.5]])).numpy())
[[-2.5144 -0.915 -1.0911 -2.0841]
[-4.0144 -2.415 -2.5911 -3.5841]]
Source code in tinygrad/tensor.py
3653 3654 3655 3656 3657 3658 3659 3660 3661 3662 3663 3664 3665 3666 3667 3668 3669 3670 3671 3672 |
|
mul
¤
Multiplies self
and x
.
Equivalent to self * x
.
Supports broadcasting to a common shape, type promotion, and integer, float, boolean inputs.
Tensor.manual_seed(42)
t = Tensor.randn(4)
print(t.numpy())
[-0.5144 1.085 0.9089 -0.0841]
print(t.mul(3).numpy())
[-1.5431 3.2549 2.7267 -0.2523]
print(t.mul(Tensor([[-1.0], [2.0]])).numpy())
[[ 0.5144 -1.085 -0.9089 0.0841]
[-1.0287 2.17 1.8178 -0.1682]]
Source code in tinygrad/uop/mathtraits.py
41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
|
div
¤
div(
x: Tensor | ConstType,
reverse=False,
rounding_mode: Literal["trunc", "floor"] | None = None,
) -> Tensor
Divides self
by x
.
Equivalent to self / x
.
Supports broadcasting to a common shape, type promotion, and integer, float, boolean inputs.
div
performs true division.
Tensor.manual_seed(42)
t = Tensor.randn(4)
print(t.numpy())
[-0.5144 1.085 0.9089 -0.0841]
print(t.div(3).numpy())
[-0.1715 0.3617 0.303 -0.028 ]
print(Tensor([1, 4, 10]).div(Tensor([2, 3, 4])).numpy())
[0.5 1.3333 2.5 ]
Source code in tinygrad/tensor.py
3674 3675 3676 3677 3678 3679 3680 3681 3682 3683 3684 3685 3686 3687 3688 3689 3690 3691 3692 3693 3694 3695 3696 3697 3698 3699 3700 3701 3702 3703 3704 3705 3706 |
|
idiv
¤
Divides self
by x
.
Equivalent to self // x
.
Supports broadcasting to a common shape, type promotion, and integer inputs.
idiv
performs integer division (truncate towards zero).
print(Tensor([-4, 7, 5, 4, -7, 8]).idiv(Tensor([2, -3, 8, -2, 3, 5])).numpy())
[-2 -2 0 -2 -2 1]
Source code in tinygrad/uop/mathtraits.py
103 104 105 106 107 108 109 110 111 112 113 114 |
|
mod
¤
Mod self
by x
.
Equivalent to self % x
.
Supports broadcasting to a common shape, type promotion, and integer inputs.
print(Tensor([-4, 7, 5, 4, -7, 8]).mod(Tensor([2, -3, 8, -2, 3, 5])).numpy())
[ 0 -2 5 0 2 3]
Source code in tinygrad/tensor.py
3708 3709 3710 3711 3712 3713 3714 3715 3716 3717 3718 3719 |
|
bitwise_xor
¤
Computes bitwise xor of self
and x
.
Equivalent to self ^ x
.
Supports broadcasting to a common shape, type promotion, and integer, boolean inputs.
print(Tensor([-1, -2, 3]).bitwise_xor(Tensor([1, 0, 3])).numpy())
[-2 -2 0]
print(Tensor([True, True, False, False]).bitwise_xor(Tensor([True, False, True, False])).numpy())
[False True True False]
Source code in tinygrad/uop/mathtraits.py
88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 |
|
bitwise_and
¤
Computes the bitwise AND of self
and x
.
Equivalent to self & x
.
Supports broadcasting to a common shape, type promotion, and integer, boolean inputs.
print(Tensor([2, 5, 255]).bitwise_and(Tensor([3, 14, 16])).numpy())
[ 2 4 16]
print(Tensor([True, True, False, False]).bitwise_and(Tensor([True, False, True, False])).numpy())
[ True False False False]
Source code in tinygrad/uop/mathtraits.py
60 61 62 63 64 65 66 67 68 69 70 71 72 73 |
|
bitwise_or
¤
Computes the bitwise OR of self
and x
.
Equivalent to self | x
.
Supports broadcasting to a common shape, type promotion, and integer, boolean inputs.
print(Tensor([2, 5, 255]).bitwise_or(Tensor([4, 4, 4])).numpy())
[ 6 5 255]
print(Tensor([True, True, False, False]).bitwise_or(Tensor([True, False, True, False])).numpy())
[ True True True False]
Source code in tinygrad/uop/mathtraits.py
74 75 76 77 78 79 80 81 82 83 84 85 86 87 |
|
bitwise_not
¤
bitwise_not() -> Tensor
Computes the bitwise NOT of self
.
Equivalent to ~self
.
print(Tensor([0, 2, 5, 255], dtype="int8").bitwise_not().numpy())
[-1 -3 -6 0]
print(Tensor([True, False]).bitwise_not().numpy())
[False True]
Source code in tinygrad/tensor.py
3721 3722 3723 3724 3725 3726 3727 3728 3729 3730 3731 3732 3733 |
|
lshift
¤
Computes left arithmetic shift of self
by x
bits. self
must have unsigned dtype.
Equivalent to self << x
.
print(Tensor([1, 3, 31], dtype=dtypes.uint8).lshift(2).numpy())
[ 4 12 124]
Source code in tinygrad/tensor.py
3735 3736 3737 3738 3739 3740 3741 3742 3743 3744 3745 |
|
rshift
¤
Computes right arithmetic shift of self
by x
bits. self
must have unsigned dtype.
Equivalent to self >> x
.
print(Tensor([4, 13, 125], dtype=dtypes.uint8).rshift(2).numpy())
[ 1 3 31]
Source code in tinygrad/tensor.py
3747 3748 3749 3750 3751 3752 3753 3754 3755 3756 3757 |
|
pow
¤
Computes power of self
with x
.
Equivalent to self ** x
.
print(Tensor([-1, 2, 3]).pow(2.0).numpy())
[1 4 9]
print(Tensor([-1, 2, 3]).pow(Tensor([-1.5, 0.5, 1.5])).numpy())
[-2147483648 1 5]
print((2.0 ** Tensor([-1, 2, 3])).numpy())
[0.5 4. 8. ]
Source code in tinygrad/tensor.py
3759 3760 3761 3762 3763 3764 3765 3766 3767 3768 3769 3770 3771 3772 3773 3774 3775 3776 3777 3778 3779 3780 |
|
maximum
¤
Computes element-wise maximum of self
and x
.
print(Tensor([-1, 2, 3]).maximum(1).numpy())
[1 2 3]
print(Tensor([-1, 2, 3]).maximum(Tensor([-4, -2, 9])).numpy())
[-1 2 9]
Source code in tinygrad/tensor.py
3782 3783 3784 3785 3786 3787 3788 3789 3790 3791 3792 3793 |
|
minimum
¤
Computes element-wise minimum of self
and x
.
print(Tensor([-1, 2, 3]).minimum(1).numpy())
[-1 1 1]
print(Tensor([-1, 2, 3]).minimum(Tensor([-4, -2, 9])).numpy())
[-4 -2 3]
Source code in tinygrad/tensor.py
3795 3796 3797 3798 3799 3800 3801 3802 3803 3804 3805 3806 3807 |
|
where
¤
Returns a tensor of elements selected from either x
or y
, depending on self
.
output_i = x_i if self_i else y_i
.
cond = Tensor([[True, True, False], [True, False, False]])
print(cond.where(1, 3).numpy())
[[1 1 3]
[1 3 3]]
Tensor.manual_seed(42)
cond = Tensor.randn(2, 3)
print(cond.numpy())
[[ 0.9779 0.4678 0.5526]
[-0.3288 -0.8555 0.2753]]
print((cond > 0).where(cond, -float("inf")).numpy())
[[0.9779 0.4678 0.5526]
[ -inf -inf 0.2753]]
Source code in tinygrad/tensor.py
3809 3810 3811 3812 3813 3814 3815 3816 3817 3818 3819 3820 3821 3822 3823 3824 3825 3826 3827 3828 3829 3830 3831 |
|
copysign
¤
copysign(other) -> Tensor
Returns a tensor of with the magnitude of self
and the sign of other
, elementwise.
Source code in tinygrad/tensor.py
3833 3834 3835 3836 3837 3838 3839 3840 |
|
logaddexp
¤
logaddexp(other) -> Tensor
Calculates (self.exp()+other.exp()).log(), elementwise.
Source code in tinygrad/tensor.py
3842 3843 3844 3845 3846 3847 |
|
Casting Ops¤
cast
¤
cast(dtype: DTypeLike) -> Tensor
Casts self
to the given dtype
.
t = Tensor([-1, 2.5, 3], dtype=dtypes.float)
print(t.dtype, t.numpy())
dtypes.float [-1. 2.5 3. ]
t = t.cast(dtypes.int32)
print(t.dtype, t.numpy())
dtypes.int [-1 2 3]
t = t.cast(dtypes.uint8)
print(t.dtype, t.numpy())
dtypes.uchar [255 2 3]
Source code in tinygrad/tensor.py
4298 4299 4300 4301 4302 4303 4304 4305 4306 4307 4308 4309 4310 4311 4312 4313 4314 4315 4316 4317 4318 |
|
bitcast
¤
bitcast(dtype: DTypeLike) -> Tensor
Bitcasts self
to the given dtype
of the same itemsize.
self
must not require a gradient.
t = Tensor([-1, 2, 3], dtype=dtypes.int32)
print(t.dtype, t.numpy())
dtypes.int [-1 2 3]
t = t.bitcast(dtypes.uint32)
print(t.dtype, t.numpy())
dtypes.uint [4294967295 2 3]
Source code in tinygrad/tensor.py
4320 4321 4322 4323 4324 4325 4326 4327 4328 4329 4330 4331 4332 4333 4334 4335 4336 4337 4338 4339 4340 4341 4342 4343 4344 4345 4346 |
|
float
¤
float() -> Tensor
Convenience method to cast self
to a float32
Tensor.
t = Tensor([-1, 2, 3], dtype=dtypes.int32)
print(t.dtype, t.numpy())
dtypes.int [-1 2 3]
t = t.float()
print(t.dtype, t.numpy())
dtypes.float [-1. 2. 3.]
Source code in tinygrad/tensor.py
4348 4349 4350 4351 4352 4353 4354 4355 4356 4357 4358 4359 4360 4361 |
|
half
¤
half() -> Tensor
Convenience method to cast self
to a float16
Tensor.
t = Tensor([-1, 2, 3], dtype=dtypes.int32)
print(t.dtype, t.numpy())
dtypes.int [-1 2 3]
t = t.half()
print(t.dtype, t.numpy())
dtypes.half [-1. 2. 3.]
Source code in tinygrad/tensor.py
4363 4364 4365 4366 4367 4368 4369 4370 4371 4372 4373 4374 4375 4376 |
|
int
¤
int() -> Tensor
Convenience method to cast self
to a int32
Tensor.
t = Tensor([-1.5, -0.5, 0.0, 0.5, 1.5])
print(t.dtype, t.numpy())
dtypes.float [-1.5 -0.5 0. 0.5 1.5]
t = t.int()
print(t.dtype, t.numpy())
dtypes.int [-1 0 0 0 1]
Source code in tinygrad/tensor.py
4378 4379 4380 4381 4382 4383 4384 4385 4386 4387 4388 4389 4390 4391 |
|
bool
¤
bool() -> Tensor
Convenience method to cast self
to a bool
Tensor.
t = Tensor([-1, 0, 1])
print(t.dtype, t.numpy())
dtypes.int [-1 0 1]
t = t.bool()
print(t.dtype, t.numpy())
dtypes.bool [ True False True]
Source code in tinygrad/tensor.py
4393 4394 4395 4396 4397 4398 4399 4400 4401 4402 4403 4404 4405 4406 |
|