o H&i@sddlZddlZddlZzddlZdZWn eydZYnwddlmZgdZ Gdddej j j Z ddZ ddd d d Zd d ZdS)NT)Any)autocast custom_fwd custom_bwdcsleZdZdZdejdfdedejdeffdd Zfdd Z d e d e d e ffd d Z fddZ Z S)rzSee :class:`torch.autocast`. ``torch.cuda.amp.autocast(args...)`` is equivalent to ``torch.autocast("cuda", args...)`` Tenableddtype cache_enabledcs8tjr||_d|_||_dStjd|||ddS)Ncuda)rrr)torch _jit_internal is_scriptingZ_enabledZdeviceZ fast_dtypesuper__init__)selfrrr __class__GC:\wamp64\www\opt\env\Lib\site-packages\torch/cuda/amp/autocast_mode.pyrs  zautocast.__init__cstjr|StSN)r r r r __enter__)rrrrr&s  zautocast.__enter__exc_typeexc_valexc_tbcstjrdSt|||Sr)r r r r __exit__)rrrrrrrr,s zautocast.__exit__cstjr|St|Sr)r r r r __call__)rfuncrrrr1s  zautocast.__call__)__name__ __module__ __qualname____doc__r Zfloat16boolrrrrrr __classcell__rrrrrs rcst|tjr|o|jo|jtju}|r|S|St|tt fr%|St r/t|t j r/|St|t jjrAfdd|DSt|t jjr`fdd|D}t|ttfr^t||S|S|S)Ncs"i|] \}}t|t|qSr_cast).0kvrrr Fs"z_cast..c3s|]}t|VqdSrr")r$r&r'rr Hsz_cast..) isinstancer ZTensorZis_floating_pointZis_cudarZfloat64tostrbytes HAS_NUMPYnpZndarray collectionsabcMappingitemsIterablelisttupletype)valuerZ is_eligibleiterablerr'rr#9s&   r# cast_inputscs2dur tjtdStfdd}|S)aa Create a helper decorator for ``forward`` methods of custom autograd functions. Autograd functions are subclasses of :class:`torch.autograd.Function`. See the :ref:`example page` for more detail. Args: cast_inputs (:class:`torch.dtype` or None, optional, default=None): If not ``None``, when ``forward`` runs in an autocast-enabled region, casts incoming floating-point CUDA Tensors to the target dtype (non-floating-point Tensors are not affected), then executes ``forward`` with autocast disabled. If ``None``, ``forward``'s internal ops execute with the current autocast state. .. note:: If the decorated ``forward`` is called outside an autocast-enabled region, :func:`custom_fwd` is a no-op and ``cast_inputs`` has no effect. Nr:cst|d_durt|d_|i|St}d|d_|rHtddt|it|WdS1sAwYdS|i|S)NrF)r)r Zget_autocast_gpu_dtype_dtypeZis_autocast_enabled_fwd_used_autocastrr#)argskwargsZautocast_contextr;fwdrr decorate_fwdns  $z custom_fwd..decorate_fwd) functoolspartialrwraps)rAr;rBrr@rrYs rcstfdd}|S)a7Create a helper decorator for backward methods of custom autograd functions. Autograd functions are subclasses of :class:`torch.autograd.Function`. Ensures that ``backward`` executes with the same autocast state as ``forward``. See the :ref:`example page` for more detail. csJt|dj|djd|i|WdS1swYdS)Nr)rr)rr=r<)r>r?bwdrr decorate_bwds $z custom_bwd..decorate_bwd)rCrE)rGrHrrFrrsrr)r0rCr numpyr/r.ModuleNotFoundErrortypingr__all__ampZ autocast_moderr#rrrrrrs  ( *