o <&i[@s|dZddlZddlZddlmZmZmZmZmZm Z m Z m Z m Z m Z ddlZer.edddlZddlmZgdZe dZe dZe d Ze d ZeZeZeZeege eeeffZeeeegefZeeeegefZeZeegefZeegefZ d ed efd dZ!ddddde edededee"deedee d dfddZ#ddddde edededee"deedee d dfddZ$ddddde edededee"deedee d dfddZ%ded e eeeffddZ&deed ed efd!d"Z'ded eefd#d$Z(ded efd%d&Z)d ed'efded efd(d)Z*d ed'efded efd*d+Z+e e ee efZ,e e ee ee efZ-e e ee e ed'ffZ.ee eefgefZ/ee eeefgefZ0eegefZ1eegefZ2eegeegeffZ3e d,e,eefd e3e/eeeffd-d.Z4e d,e-eeefd e3e0eeeeffd/d.Z4e d,e ed e3e1eeffd0d.Z4e d,e.d e3e2efd1d.Z4d,e.d e3e2efd2d.Z4e d,e ed e1eefded efd3d4Z5e d,e,eefd e/eeefded efd5d4Z5e d,e-eeefd e0eeeefded efd6d4Z5d,e.d e2eded efd7d4Z5e d,e ed e1eefded efd8d9Z6e d,e,eefd e/eeefded efd:d9Z6e d,e-eeefd e0eeeefded efd;d9Z6d,e.d e2eded efdd?Z8d=eege7fded e7fd@dAZ9e d,e ed=e1ee7fded e7fdBdCZ:e d,e,eefd=e/eee7fded e7fdDdCZ:e d,e-eeefd=e0eeee7fded e7fdEdCZ:d,e.d=e2e7ded e7fdFdCZ:e d,e ed=e1ee7fded e7fdGdHZ;e d,e,eefd=e/eee7fded e7fdIdHZ;e d,e-eeefd=e0eeee7fded e7fdJdHZ;d,e.d=e2e7ded e7fdKdHZ;dLedMed eefdNdOZd e"fdSdTZ?dUe"d efdVdWZ@GdXdYdYZAd ed e"fdZd[ZBGd\d]d]eCeZDGd^d_d_eeDd`ZEdS)ba Contains utility functions for working with nested python data structures. A *pytree* is Python nested data structure. It is a tree in the sense that nodes are Python collections (e.g., list, tuple, dict) and the leaves are Python values. Furthermore, a pytree should not contain reference cycles. pytrees are useful for working with nested collections of Tensors. For example, one can use `tree_map` to map a function over all Tensors inside some nested collection of Tensors and `tree_leaves` to get a flat list of all Tensors inside some nested collection. pytrees are helpful for implementing nested collection support for PyTorch APIs. N) AnyCallableIterableListOptionaloverloadTupleTypeTypeVarUnionz4C++ pytree utilities do not work with torch::deploy.) PyTreeSpec)PyTreeContext FlattenFunc UnflattenFuncDumpableContextToDumpableContextFnFromDumpableContextFnTreeSpecLeafSpecregister_pytree_node tree_flattentree_unflatten tree_leavestree_structuretree_map tree_map_ tree_map_onlytree_map_only_tree_alltree_any tree_all_only tree_any_onlytreespec_dumpstreespec_loadstreespec_pprintTSURfuncreturncs(tdtdtdtffdd }|S)Nargskwargsr+cst|i|SN)reversed)r,r-r*BC:\wamp64\www\opt\env\Lib\site-packages\torch/utils/_cxx_pytree.pywrappedVsz_reverse_args..wrapped) functoolswrapsrr*r3r1r0r2 _reverse_argsUsr7serialized_type_nameto_dumpable_contextfrom_dumpable_contextcls flatten_fn unflatten_fnr9r:r;cCs:t||||||dddlm}|j||||||ddS)aRegister a container-like type as pytree node. Args: cls (type): A Python type to treat as an internal pytree node. flatten_fn (callable): A function to be used during flattening, taking an instance of ``cls`` and returning a pair, with (1) an iterable for the children to be flattened recursively, and (2) some hashable auxiliary data to be stored in the treespec and to be passed to the ``unflatten_fn``. unflatten_fn (callable): A function taking two arguments: the auxiliary data that was returned by ``flatten_fn`` and stored in the treespec, and the unflattened children. The function should return an instance of ``cls``. serialized_type_name (str, optional): A keyword argument used to specify the fully qualified name used when serializing the tree spec. to_dumpable_context (callable, optional): An optional keyword argument to custom specify how to convert the context of the pytree to a custom json dumpable representation. This is used for json serialization, which is being used in :mod:`torch.export` right now. from_dumpable_context (callable, optional): An optional keyword argument to custom specify how to convert the custom json dumpable representation of the context back to the original context. This is used for json deserialization, which is being used in :mod:`torch.export` right now. Example:: >>> # xdoctest: +SKIP >>> # Registry a Python type with lambda functions >>> register_pytree_node( ... set, ... lambda s: (sorted(s), None, None), ... lambda children, _: set(children), ... ) r8)_pytreeN)_private_register_pytree_noder@)r<r=r>r9r:r;pythonr1r1r2r]s"(  rcCs&tjdddt||||||ddS)aRegister a container-like type as pytree node for the C++ pytree only. The ``namespace`` argument is used to avoid collisions that occur when different libraries register the same Python type with different behaviors. It is recommended to add a unique prefix to the namespace to avoid conflicts with other libraries. Namespaces can also be used to specify the same class in different namespaces for different use cases. .. warning:: For safety reasons, a ``namespace`` must be specified while registering a custom type. It is used to isolate the behavior of flattening and unflattening a pytree node type. This is to prevent accidental collisions between different libraries that may register the same type. Args: cls (type): A Python type to treat as an internal pytree node. flatten_fn (callable): A function to be used during flattening, taking an instance of ``cls`` and returning a pair, with (1) an iterable for the children to be flattened recursively, and (2) some hashable auxiliary data to be stored in the treespec and to be passed to the ``unflatten_fn``. unflatten_fn (callable): A function taking two arguments: the auxiliary data that was returned by ``flatten_fn`` and stored in the treespec, and the unflattened children. The function should return an instance of ``cls``. serialized_type_name (str, optional): A keyword argument used to specify the fully qualified name used when serializing the tree spec. to_dumpable_context (callable, optional): An optional keyword argument to custom specify how to convert the context of the pytree to a custom json dumpable representation. This is used for json serialization, which is being used in :mod:`torch.export` right now. from_dumpable_context (callable, optional): An optional keyword argument to custom specify how to convert the custom json dumpable representation of the context back to the original context. This is used for json deserialization, which is being used in :mod:`torch.export` right now. z}torch.utils._cxx_pytree._register_pytree_node is deprecated. Please use torch.utils._cxx_pytree.register_pytree_node instead.) stacklevelr8N)warningswarnrAr<r=r>r9r:r;r1r1r2_register_pytree_nodes( rIcCs(t|stj||t|dddSdS)zThis is an internal function that is used to register a pytree node type for the C++ pytree only. End-users should use :func:`register_pytree_node` instead. torch) namespaceN)optreeZis_structseq_classrr7rHr1r1r2rAs  rAtreecCtj|dddS)aFlatten a pytree. See also :func:`tree_unflatten`. The flattening order (i.e., the order of elements in the output list) is deterministic, corresponding to a left-to-right depth-first tree traversal. >>> tree = {'b': (2, [3, 4]), 'a': 1, 'c': None, 'd': 5} >>> tree_flatten(tree) ([1, 2, 3, 4, None, 5], PyTreeSpec({'a': *, 'b': (*, [*, *]), 'c': *, 'd': *}, NoneIsLeaf)) >>> tree_flatten(1) ([1], PyTreeSpec(*, NoneIsLeaf)) >>> tree_flatten(None) ([None], PyTreeSpec(*, NoneIsLeaf)) For unordered dictionaries, :class:`dict` and :class:`collections.defaultdict`, the order is dependent on the **sorted** keys in the dictionary. Please use :class:`collections.OrderedDict` if you want to keep the keys in the insertion order. >>> from collections import OrderedDict >>> tree = OrderedDict([('b', (2, [3, 4])), ('a', 1), ('c', None), ('d', 5)]) >>> tree_flatten(tree) ([2, 3, 4, 1, None, 5], PyTreeSpec(OrderedDict([('b', (*, [*, *])), ('a', *), ('c', *), ('d', *)]), NoneIsLeaf)) Args: tree (pytree): A pytree to flatten. Returns: A pair ``(leaves, treespec)`` where the first element is a list of leaf values and the second element is a treespec representing the structure of the pytree. TrJ none_is_leafrK)rLrrMr1r1r2rs  rleavestreespeccCs*t|tstdt|dt||S)adReconstruct a pytree from the treespec and the leaves. The inverse of :func:`tree_flatten`. >>> tree = {'b': (2, [3, 4]), 'a': 1, 'c': None, 'd': 5} >>> leaves, treespec = tree_flatten(tree) >>> tree == tree_unflatten(leaves, treespec) True Args: leaves (iterable): The list of leaves to use for reconstruction. The list must match the number of leaves of the treespec. treespec (TreeSpec): The treespec to reconstruct. Returns: The reconstructed pytree, containing the ``leaves`` placed in the structure described by ``treespec``. z^tree_unflatten(values, spec): Expected `spec` to be instance of TreeSpec but got item of type .) isinstancer TypeErrortyperLr)rRrSr1r1r2rs  rcCrN)a^Get the leaves of a pytree. See also :func:`tree_flatten`. >>> tree = {'b': (2, [3, 4]), 'a': 1, 'c': None, 'd': 5} >>> tree_leaves(tree) [1, 2, 3, 4, None, 5] >>> tree_leaves(1) [1] >>> tree_leaves(None) [None] Args: tree (pytree): A pytree to flatten. Returns: A list of leaf values. TrJrO)rLrrQr1r1r2r,srcCrN)aGet the treespec for a pytree. See also :func:`tree_flatten`. >>> tree = {'b': (2, [3, 4]), 'a': 1, 'c': None, 'd': 5} >>> tree_structure(tree) PyTreeSpec({'a': *, 'b': (*, [*, *]), 'c': *, 'd': *}, NoneIsLeaf) >>> tree_structure(1) PyTreeSpec(*, NoneIsLeaf) >>> tree_structure(None) PyTreeSpec(*, NoneIsLeaf) Args: tree (pytree): A pytree to flatten. Returns: A treespec object representing the structure of the pytree. TrJrO)rLrrQr1r1r2rBs r.cCtj||dddS)a Map a function over leaves in a pytree to produce a new pytree. See also :func:`tree_map_`. >>> tree_map(lambda x: x + 1, {'x': 7, 'y': (42, 64)}) {'x': 8, 'y': (43, 65)} >>> tree_map(lambda x: x is None, {'x': 7, 'y': (42, 64), 'z': None}) {'x': False, 'y': (False, False), 'z': True} Args: func (callable): A function that takes a single argument, to be applied at the corresponding leaves of the pytree. tree (pytree): A pytree to be mapped over, with each leaf providing the argument to function ``func``. Returns: A new pytree with the same structure as ``tree`` but with the value at each leaf given by ``func(x)`` where ``x`` is the value at the corresponding leaf in ``tree``. TrJrO)rLrr*rMr1r1r2r\s rcCrX)amLike :func:`tree_map`, but do an inplace call on each leaf and return the original tree. See also :func:`tree_map`. Args: func (callable): A function that takes a single argument, to be applied at the corresponding leaves of the pytree. tree (pytree): A pytree to be mapped over, with each leaf providing the argument to function ``func``. Returns: The original ``tree`` with the value at each leaf is given by the side-effect of function ``func(x)`` (not the return value) where ``x`` is the value at the corresponding leaf in ``tree``. TrJrO)rLrrYr1r1r2rxs r__type_or_typescCdSr.r1rZr1r1r2map_onlyr]cCr[r.r1r\r1r1r2r]r^cCr[r.r1r\r1r1r2r]r^cCr[r.r1r\r1r1r2r]r^cs.dttgtfdttgtfffdd }|S)a Suppose you are writing a tree_map over tensors, leaving everything else unchanged. Ordinarily you would have to write: def go(t): if isinstance(t, Tensor): return ... else: return t With this function, you only need to write: @map_only(Tensor) def go(t): return ... You can also directly use 'tree_map_only' r*r+cs&tdtdtffdd }|S)Nxr+cst|r |S|Sr.rU)r_)rZr*r1r2r3s z*map_only..wrapper..wrapped)r4r5r&rr6r\r0r2wrapperszmap_only..wrapper)rr&r)rZrar1r\r2r]s* cCr[r.r1rZr*rMr1r1r2rrcCr[r.r1rbr1r1r2rrccCr[r.r1rbr1r1r2rrccCtt|||Sr.)rr]rbr1r1r2rcCr[r.r1rbr1r1r2rrcrcCr[r.r1rbr1r1r2rrccCr[r.r1rbr1r1r2rrccCrdr.)rr]rbr1r1r2rrepredcCt|}tt||Sr.)rallmaprfrM flat_argsr1r1r2rrcCrgr.)ranyrirjr1r1r2r rlr cCr[r.r1rZrfrMr1r1r2r!#rcr!cCr[r.r1rnr1r1r2r!,rccCr[r.r1rnr1r1r2r!5rcc t|}tfdd|DS)Nc3"|] }t|r|VqdSr.r`.0r_rZrfr1r2 D z tree_all_only..)rrhrZrfrMrkr1rsr2r!>cCr[r.r1rnr1r1r2r"Grcr"cCr[r.r1rnr1r1r2r"PrccCr[r.r1rnr1r1r2r"Yrccro)Nc3rpr.r`rqrsr1r2rthruz tree_any_only..)rrmrvr1rsr2r"brw prefix_tree full_treecCrX)agReturn a list of broadcasted leaves in ``prefix_tree`` to match the number of leaves in ``full_tree``. If a ``prefix_tree`` is a prefix of a ``full_tree``, this means the ``full_tree`` can be constructed by replacing the leaves of ``prefix_tree`` with appropriate **subtrees**. This function returns a list of leaves with the same size as ``full_tree``. The leaves are replicated from ``prefix_tree``. The number of replicas is determined by the corresponding subtree in ``full_tree``. >>> broadcast_prefix(1, [1, 2, 3]) [1, 1, 1] >>> broadcast_prefix([1, 2, 3], [1, 2, 3]) [1, 2, 3] >>> broadcast_prefix([1, 2, 3], [1, 2, 3, 4]) Traceback (most recent call last): ... ValueError: list arity mismatch; expected: 3, got: 4; list: [1, 2, 3, 4]. >>> broadcast_prefix([1, 2, 3], [1, 2, (3, 4)]) [1, 2, 3, 3] >>> broadcast_prefix([1, 2, 3], [1, 2, {'a': 3, 'b': 4, 'c': (None, 5)}]) [1, 2, 3, 3, 3, 3] Args: prefix_tree (pytree): A pytree with the same structure as a prefix of ``full_tree``. full_tree (pytree): A pytree with the same structure as a suffix of ``prefix_tree``. is_leaf (callable, optional): An optionally specified function that will be called at each flattening step. It should return a boolean, with :data:`True` stopping the traversal and the whole subtree being treated as a leaf, and :data:`False` indicating the flattening should traverse the current object. Returns: A list of leaves in ``prefix_tree`` broadcasted to match the number of leaves in ``full_tree``. TrJrO)rLbroadcast_prefix)rxryr1r1r2rzks "rzcCsBt|tsJtdg|j|}zt||WSty YdSw)Nr)rUrr num_leavesrz ValueError)rMrSryr1r1r2_broadcast_to_and_flattens  r}protocolcCsPt|tstdt|dddlm}m}|tdg|j|}|||dS)z&Serialize a treespec to a JSON string.zVtreespec_dumps(spec): Expected `spec` to be instance of TreeSpec but got item of type rTr?)rr#r)r~) rUrrVrWr@rr#rr{)rSr~Z_tree_structureZ_treespec_dumps orig_treespecr1r1r2r#s  r# serializedcCs6ddlm}m}||}|dg|j|}t|}|S)z*Deserialize a treespec from a JSON string.r?)rr$r)r@rr$r{r)rZ_tree_unflattenZ_treespec_loadsr dummy_treerSr1r1r2r$s r$c@seZdZdefddZdS) _DummyLeafr+cCsdS)N*r1)selfr1r1r2__repr__sz_DummyLeaf.__repr__N)__name__ __module__ __qualname__strrr1r1r1r2rsrcCs"tddt|jD|}t|S)NcSsg|]}tqSr1)r)rr_r1r1r2 sz#treespec_pprint..)rranger{repr)rSrr1r1r2r%s r%c@seZdZdedefddZdS) LeafSpecMetainstancer+cCst|to|Sr.)rUrZis_leaf)rrr1r1r2__instancecheck__szLeafSpecMeta.__instancecheck__N)rrrobjectboolrr1r1r1r2rsrc@seZdZdddZdS)rr+cCs tjddS)NT)rP)rLZ treespec_leaf)r<r1r1r2__new__s zLeafSpec.__new__N)r+r)rrrrr1r1r1r2rsr) metaclassr.)F__doc__r4rFtypingrrrrrrrr r r rJZ_running_with_deploy ImportErrorrLr __all__r&r'r(r)rr rrrZOpTreeUnflattenFuncrrrr7rrrIrArrrrrrZType2ZType3ZTypeAnyZFn2ZFn3ZFnZFnAnyZ MapOnlyFnr]rrrrr r!r"rzr}intr#r$rr%rWrrr1r1r1r2s0    B = '*.$                                    2