B Khh4@sdZddlZddlZddlZddlZddlZddlmZddlm Z m Z m Z m Z ej dkrhddlmZn ddlmZddlmZddlmZed Zed Zed ZejZejd d GdddZeeeedddZee eefdddZeeedddZ ee ee efdddZ!eeedddZ"ee ee efdddZ#ee ee efdd d!Z$ee eed"d#d$Z%ej&eej'd%d&d'Z(e ej&e ed(d)d*Z)ejd d Gd+d,d,Z*Gd-d.d.ej+Z,ejd d Gd/d0d0Z-Gd1d2d2ej+Z.dS)3z)Functions to process IPython magics with.N) lru_cache)DictListOptionalTuple) ) TypeGuard)out)NothingChanged)zget_ipython().run_cell_magiczget_ipython().systemzget_ipython().getoutputzget_ipython().run_line_magic) ENDMARKERNLNEWLINECOMMENTDEDENTZUNIMPORTANT_WSZ ESCAPED_NL)captureZprunZpypypythonpython3timeZtimeitT)frozenc@seZdZUeed<eed<dS) ReplacementmasksrcN)__name__ __module__ __qualname__str__annotations__rrq*|jdkr\|jdkr\||=d}Pq*W|sn|dfS||dfS)aRemove trailing semicolon from Jupyter notebook cell. For example, fig, ax = plt.subplots() ax.plot(x_data, y_data); # plot data would become fig, ax = plt.subplots() ax.plot(x_data, y_data) # plot data Mirrors the logic in `quiet` from `IPython.core.displayhook`, but uses ``tokenize_rt`` so that round-tripping works fine. r)reversed_enumerate src_to_tokens tokens_to_srcFOP;T)r$r(r)r*nameTOKENS_TO_IGNOREr)rr(r)r*tokensZtrailing_semicolonidxtokenrrrremove_trailing_semicolonKs r2)rhas_trailing_semicolonr"cCst|s|Sddlm}m}m}||}xB||D],\}}|jtkrBq.|j|jdd||<Pq.Wtddt ||S)zPut trailing semicolon back if cell originally had it. Mirrors the logic in `quiet` from `IPython.core.displayhook`, but uses ``tokenize_rt`` so that round-tripping works fine. r)r(r)r*r,)rz{INTERNAL ERROR: Was not able to reinstate trailing semicolon. Please report a bug on https://github.com/psf/black/issues. N) r$r(r)r*r-r._replacerAssertionErrorr)rr3r(r)r*r/r0r1rrrput_trailing_semicolon_backks r6cCsg}yt|Wntk r&Yn X||fSddlm}|}||}t|\}}||7}||}t|\}}t| t| krt ||7}||fS)zMask IPython magics so content becomes parseable Python code. For example, %matplotlib inline 'foo' becomes "25716f358c32750e" 'foo' The replacements are returned, along with the transformed code. r)TransformerManager) astparse SyntaxErrorZIPython.core.inputtransformer2r7Ztransform_cellreplace_cell_magicsreplace_magicslen splitlinesr )r replacementsr7Ztransformer_managerZ transformedZcell_magic_replacementsZmagic_replacementsrrr mask_cells"     r@)rmagicr"cCs|sttt|ddd}t|}d}x4||kr^t|}|d7}|dkr,td|dq,Wt|dt|kr~|d}d|dS) aDReturn randomly generated token to mask IPython magic with. For example, if 'magic' was `%matplotlib inline`, then a possible token to mask it with would be `"43fdd17f7e5ddc83"`. The token will be the same length as the magic, and we make sure that it was not already present anywhere else in the cell. rdzINTERNAL ERROR: Black was not able to replace IPython magic. Please report a bug on https://github.com/psf/black/issues. The magic might be helpful: N.")r5maxr= TOKEN_HEX)rrAnbytesr1counterrrr get_tokens   rKcCsjg}t|}t}|||jdkr0||fS|jj}t||}|t||d|d|jj |fS)aNReplace cell magic with token. Note that 'src' will already have been processed by IPython's TransformerManager().transform_cell. Example, get_ipython().run_cell_magic('t', '-n1', 'ls =!ls\n') becomes "a794." ls =!ls The replacement, along with the transformed code, is returned. N)rr ) r8r9CellMagicFindervisit cell_magicheaderrKappendrbody)rr?treeZcell_magic_finderrPrrrrr;s    r;c Csg}t}|t|g}xt|ddD]\}}||jkr|j|}t|dkrhtd|d|dj |dj }}t ||} | t | |d|d|| }| |q0Wd||fS) aoReplace magics within body of cell. Note that 'src' will already have been processed by IPython's TransformerManager().transform_cell. Example, this get_ipython().run_line_magic('matplotlib', 'inline') 'foo' becomes "5e67db56d490fd39" 'foo' The replacement, along with the transformed code, are returned. rC)startz#Expecting one magic per line, got: z< Please report a bug on https://github.com/psf/black/issues.r)rrNrL) MagicFinderrNr8r9 enumerater>magicsr=r5 col_offsetrArKrQrjoin) rr?Z magic_finderZnew_srcsilineZoffsets_and_magicsrXrArrrrr<s"    r<)rr?r"cCs"x|D]}||j|j}qW|S)zRemove replacements from cell. For example "9b20" foo = bar becomes %%time foo = bar )replacerr)rr? replacementrrr unmask_cell s r^)noder"cCs8t|tjo6t|jtjo6t|jjtjo6|jjjdkS)zCheck if attribute is IPython magic. Note that the source of the abstract syntax tree will already have been processed by IPython's TransformerManager().transform_cell. Z get_ipython) isinstancer8 AttributevalueCallfuncNameid)r_rrr_is_ipython_magics rg)argsr"cCs2g}x(|D] }t|tjst||jq W|S)N)r`r8Strr5rQs)rhZstr_argsargrrr _get_str_args.s  rlc@s<eZdZUeed<eeed<eed<eedddZdS) CellMagicr-paramsrR)r"cCs&|jrd|jd|jSd|jS)Nz%% )rnr-)selfrrrrP<szCellMagic.headerN)rrrrrrpropertyrPrrrrrm6s  rmc@s8eZdZdZd eeddddZejddddZ dS) rMa^Find cell magics. Note that the source of the abstract syntax tree will already have been processed by IPython's TransformerManager().transform_cell. For example, %%time foo() would have been transformed to get_ipython().run_cell_magic('time', '', 'foo()\n') and we look for instances of the latter. N)rOr"cCs ||_dS)N)rO)rprOrrr__init__VszCellMagicFinder.__init__)r_r"cCs^t|jtjrPt|jjrP|jjjdkrPt|jj}t |d|d|dd|_ | |dS)z)Find cell magic, extract header and body.Zrun_cell_magicrrCrB)r-rnrRN) r`rbr8rcrgrdattrrlrhrmrO generic_visit)rpr_rhrrr visit_ExprYs   zCellMagicFinder.visit_Expr)N) rrr__doc__rrmrrr8ExprrurrrrrMDsrMc@seZdZUeed<eed<dS)OffsetAndMagicrXrAN)rrrintrrrrrrrxes rxc@sBeZdZdZddddZejddddZejddd d Z dS) rUaVisit cell to look for get_ipython calls. Note that the source of the abstract syntax tree will already have been processed by IPython's TransformerManager().transform_cell. For example, %matplotlib inline would have been transformed to get_ipython().run_line_magic('matplotlib', 'inline') and we look for instances of the latter (and likewise for other types of magics). N)r"cCstt|_dS)N) collections defaultdictlistrW)rprrrrrszMagicFinder.__init__)r_r"cCst|jtjrt|jjrt|jj}|jjjdkrDd|d}nP|jjjdkr|d|d}|dr|d|d7}nt d|jjjd d |j |jj  t |jj|||d S) a{Look for system assign magics. For example, black_version = !black --version env = %env var would have been (respectively) transformed to black_version = get_ipython().getoutput('black --version') env = get_ipython().run_line_magic('env', 'var') and we look for instances of any of the latter. getoutput!rrun_line_magic%rCrozUnexpected IPython magic zC found. Please report a bug on https://github.com/psf/black/issues.N)r`rbr8rcrgrdrlrhrsr5rWlinenorQrxrXrt)rpr_rhrrrr visit_Assigns zMagicFinder.visit_AssigncCst|jtjrt|jjrt|jj}|jjjdkr|ddkrPd|d}q|ddkrld|d}qd|d}|dr|d |d7}n@|jjjd krd |d}n"|jjjd krd |d}nt |j |jj  t |jj|||dS)aLook for magics in body of cell. For examples, !ls !!ls ?ls ??ls would (respectively) get transformed to get_ipython().system('ls') get_ipython().getoutput('ls') get_ipython().run_line_magic('pinfo', 'ls') get_ipython().run_line_magic('pinfo2', 'ls') and we look for instances of any of the latter. rrZpinfo?rCZpinfo2z??rrosystemr~r}z!!N)r`rbr8rcrgrdrlrhrsr rWrrQrxrXrt)rpr_rhrrrrrus$   zMagicFinder.visit_Expr) rrrrvrrr8ZAssignrrwrurrrrrUms!rU)/rvr8rzZ dataclassesZsecretssys functoolsrtypingrrrr version_infor Ztyping_extensionsZ black.outputr Z black.reportr frozensetZTRANSFORMED_MAGICSr.ZPYTHON_CELL_MAGICSZ token_hexrHZ dataclassrboolr'rr2r6r@rKr;r<r^exprrargrlrmZ NodeVisitorrMrxrUrrrrsF        ()!