6h RSSKJr SSKrSSKrSSKrSSKrSSKrSSKrSSKJ r J r J r J r J r JrJr SSKJr SSKJr SSKJrJrJrJrJrJrJrJrJrJ r J!r!J"r" SSK#J$r$ \RJS\RLS \RNS 0r(\RJS \RLS \RNS 0r)SFSjr*SGSjr+SHSISjjr,SJSjr-SKSjr.SLSMSjjr/SNSjr0SOSjr1SPSjr2SQSRSjjr3SSSjr4STSjr5SJSjr6"SS5r7Sr8Sr9Sr:Sr;SUSjrS#r?S$r@SVS%jrAS&rBS'rCS(rD"S)S*\ R5rF"S+S,\ R5rHSWS-jrISWS.jrJSXS/jrK\RS05rM"S1S25rNSYS3jrO\ \PrQ\\P\ \R\Q4rS"S4S55rT"S6S75rU"S8S95rV"S:S;5rW"S<S=\W5rXSZS>jrY\WS4S[S?jjrZS\S@jr[SJSAjr\S]SBjr]S^SCjr^SKSDjr_SJSEjr`g)_) annotationsN)CallableIterableIteratorOptionalSetTupleType)_) repository) bookmarksbundle2 cacheutilerror narrowspecphasespycompat requirementsscmutilstore transactionutil)nodemapi i@lcd[U5nU[R- URU5 U$)zdetermine the final set of requirement for a new stream clone this method combine the "default" requirements that a new repository would use with the constaint we get from the stream clone content. We keep local configuration choice when possible. )setrequirementsmodSTREAM_IGNORABLE_REQUIREMENTSupdate)default_requirementsstreamed_requirementsrs 7/usr/lib/python3/dist-packages/mercurial/streamclone.pynew_stream_clone_requirementsr%Cs2+,L?@@@-. c@UR[R- nU$)zthe set of requirement the new clone will have to support This is used for advertising the stream options and to generate the actual stream content.)rrr )reporequiredformatss r$r#r#Ss% OIII r&c\URnURnURnUcURS5nU(dg[ U5(agUR (agSnUR (ab[R"USS9n[URS/55n[URRS/55n[Xx-5nU(aU(dgU(aU(dg[5n URS5(a#U R[R5 S U 4$URS5n U (d/URR R#[%S55 g[U R'S 55n XR(- n U (axURR R#[%S 5S R+[-U 55-5 URR R#[%S 55 gU n S U 4$)aWhether it is possible to perform a streaming clone as part of pull. ``bundle2`` will cause the function to consider stream clone through bundle2 and only through bundle2. Returns a tuple of (supported, requirements). ``supported`` is True if streaming clone is supported and False otherwise. ``requirements`` is a set of repo requirements from the remote, or ``None`` if stream clone isn't supported. sstream-preferred)FNFsclient)rolesstreams streamreqss=warning: stream clone requested but server has them disabled ,sGwarning: stream clone requested but client is missing requirements: %s , sQ(see https://www.mercurial-scm.org/wiki/MissingRequirement for more information) T)r(remotestreamclonerequestedcapablelenheads canusebundle2 bundle2mod getrepocapsrgetremotebundle2capsbooladdrREVLOGV1_REQUIREMENTuiwarnr split supportedjoinsorted) pulloprr(r.streamrequestedbundle2supported local_capslocal_supportedremote_supportedr streamreqs missingreqss r$canperformstreamclonerI^s ;;D ]]F11O ..)<=  4yy|| ++DyA jnnY;<v77;;IrJK BC  )5L ~~i  ==>B  ?^^M2  KKNN  "  ))$/0  >>1  KKNN  ***VK01 2  KKNN  /  !  r&c,SSKJn [U5up#U(dgURnURnSnUR S5(a9UR 5nURS05R5nSSS5 URR[S55 UR 5nURS05R5nSSS5 WR5n [U 5n U S:Xa[R""[S55eU S :Xa[R""[S 55eU S :wa[R""[S 55eUR5n [%[U R'S S55upUR+5 [-XHX5 [/UR0U5UlUR3URUR0UR45UR6l[:R<"U5 [>R@"U5 U(aURBREXF5 URG5 SSS5 g!,(df  GN =f!,(df  GN=f![a! [R "[S5U 5ef=f![[(4a! [R "[S5U 5ef=f!,(df  g=f)zPossibly perform a legacy stream clone operation. Legacy stream clones are performed as part of pull but before all other operations. A legacy stream clone will not be performed if a bundle2 stream clone is supported. r  localrepoNs branchmapsstreaming all changes s stream_out'unexpected response from remote server:soperation forbidden by servers$locking the remote repository failedrs%the server sent an unknown error code )$rLrIr(r.r0commandexecutor callcommandresultr;statusr readlineint ValueErrorr ResponseErrorAbortmapr= TypeErrorlock consumev1r%rresolvestorevfsoptionsfeaturessvfsoptionsrwritereporequirementsrpost_stream_cleanup _branchcachesreplace invalidate) rArLr>rr(r. rbranchmapefplresp filecount bytecounts r$maybeperformlegacystreamclonernsk3F;I  ;;D ]]FJ ~~l##  # # %|R8??AJ& GGNN1/01    !Q ]]=" - 4 4 6 " A 1v  qykk!<=>> kk!CDEE kk!DEFF A "3a(89 $I19     &<< GGT&&    %%d+##D)     & &t 8  E& % " !  !! 8 91     " !! 8 91   s=!I?9!J3 J#>"K0CL? J J #+K1L Lc[RUR;agURR SSSS9(dg[ R "U5nU(aURR SS5$g)z5Whether streaming clones are allowed from the server.Fsservers uncompressedT) untrustedsuncompressedallowsecret)r REPO_FEATURE_STREAM_CLONEr_r; configboolr hassecret)r(secrets r$allowservergenerationrusc++4==@ 77  iD  I  d #F ww!!)-GHH r&c6URRXUS9$)Nphase obsolescence)rwalk)r(matcherrxrys r$_walkstreamfilesr|s ::??7l? KKr&c[R"5U- nUS::aSn[S5nUU[R"U5U[R"X4- 54-nURR U5 g)z@common utility to report time it took to apply the stream bundlergMbP?s5stream-cloned %d files / %s in %.1f seconds (%s/sec) N)rtimerr rmr;rT)r( start_time file_count byte_countelapsedms r$_report_transferredrsmjjlZ'G!| CDA z" z+,  A  GGNN1r&c^^^^/mSnTR5 TRRS5 [T5 [ T5HinUR 5HRnUR TRR5nU(dM1TRURU45 X- nMT Mk [T5 SSS5 [T5 TRRS[T5U4-5 TRmTRRmSUUUU4Sjjn[T5X"54$!,(df  N=f)aEmit content for version 1 of a streaming clone. This returns a 3-tuple of (file count, byte size, data iterator). The data iterator consists of N entries for each file being transferred. Each file entry starts as a line with the file name and integer size delimited by a null byte. The raw file data follows. Following the raw file data is the next file entry, or EOF. When used on the wire protocol, an additional line indicating protocol success will be prepended to the stream. This function is not responsible for adding it. This function will obtain a repository lock to ensure a consistent view of the store is captured. It therefore may raise LockError. r scanning Ns%d files, %d bytes to transfer c3`># THupT(aTRRSX4-5 S[R"U5U4-v T"USSS9nUS::aUR U5v O[ R "X!S9ShvN SSS5 M gN!,(df  M=f7f)Nssending %s (%d bytes) s%s%d srbF) auditpathilimit)r;debugr encodedirreadr filechunkiter)namesizeri debugflagentriesr(r`s r$emitrevlogdata"generatev1..emitrevlogdataYs!JD 8D<GH!6 == =dEU3r5=''$-'#11"AAA 43"B 43s0AB.2BB B B.B B+ &B.returnzIterator[bytes])r\r;r_test_sync_point_walk_1_2r|files file_sizervfsappendunencoded_path_test_sync_point_walk_3_test_sync_point_walk_4r1r`r) r( total_bytesentryfrrrrr`s ` @@@r$ generatev1r0s&GK   m$!$'%d+E[[]KK 7 9NNA$4$4i#@A,K #, % D!GGMM+s7|[.II 99D!!I B B w<n&6 66C sA2D>4D>> E c## [U5(dSv g[U5upnSv SX4-v UShvN g![Ra Sv gf=fN#7f)aEmit content for version 1 of streaming clone suitable for the wire. This is the data output from ``generatev1()`` with 2 header lines. The first line indicates overall success. The 2nd contains the file count and byte size of payload. The success line contains "0" for success, "1" for stream generation not allowed, and "2" for error locking the repository (possibly indicating a permissions error for the server process). s1 Ns2 s0 s%d %d )rurr LockError)r(rlrmits r$generatev1wireprotorjsl ! & & #-d#3 b L  - --MM ?? s/A>AAAAAAAc^^^TS:wa [S5e[T5nSR[U55mSUUU4SjjnX#"54$)aEmit content for version 1 of a stream clone bundle. The first 4 bytes of the output ("HGS1") denote this as stream clone bundle version 1. The next 2 bytes indicate the compression type. Only "UN" is currently supported. The next 16 bytes are two 64-bit big endian unsigned integers indicating file count and byte count, respectively. The next 2 bytes is a 16-bit big endian unsigned short declaring the length of the requirements string, including a trailing . The following N bytes are the requirements string, which is ASCII containing a comma-delimited list of repo requirements that are needed to support the data. The remaining content is the output of ``generatev1()`` (which may be compressed in the future). Returns a tuple of (requirements, data generator). UNz.we do not support the compression argument yetr,c3 ># Sv Tv [T5upnTRR[S5X4-5 [R "SX5v [R "S[ T5S-5v TS-v TS:XdeTRR[S5U[S 5S 9nURS 5 UHnUR[ U5S 9 Uv M! UR5 g7f) NsHGS1swriting %d bytes for %d files >QQ>Hr rbundlebytestotalunitrstep) rr;rTr structpackr1 makeprogressr! incrementcomplete)rlrmrprogresschunk compressionr(requiress r$gengeneratebundlev1..gens #-d#3 b  0 1Y4J J kk&)77kk%X!233e###77'' iL ( (  E   CJ  /K sDDr)rWr#r?r@)r(rrrrs`` @r$generatebundlev1rsO0eIJJ(.Lyy -.H6  r&c UR5 URR[S5U[R "U54-5 URR [S5U[S5S9nURS5 [R"5nSnURS5 URRURUS9 [U5GHen[US5(aUR5nOW/n U (a U SS :Xd4U RUR!S 55 U (dM)U SS :XdM4S R#U 5nUR%S S 5up['U 5n URR0(a4URR3SU [R "U 54-5 [4R6"U 5n URU SSS9n US - n[R8"XS9H,nUR;[=U5S9 U R?U5 M. SSS5 GMh SSS5 URASS9 SSS5 URC5 [EXXc5 SSS5 g![([*4a! [,R."[S 5U5ef=f!,(df  GM=f!,(df  N=f!,(df  N=f!,(df  g=f)zApply the contents from version 1 of a streaming clone file handle. This takes the output from "stream_out" and applies it to the specified repository. Like "stream_out," the status line added by the wire protocol is not handled by this function. !%d files to transfer, %s of data clonerrr) expectedcountrU r r&rrMsadding %s (%s) wT)backgroundcloserrNclearfilecache)#r\r;rTr rrmrr!r~rr`backgroundclosingrangehasattrrUrrr?r=rVrWr[rrXrrr decodedirrrr1writerfrr)r(rirlrmrstarttotal_file_countirjl_bufrrpathofprs r$r]r]s|   3 4$..34 5 77'' hKyq{(       h ',,TWWI,Ny)Ar:..KKM !##(U2Y%-?!LL4$)5U2Y%-?HHUO%&WWUA%6 "4y ww(( /49M2NN!??40D4tD(A-(%)%7%7%GE$..CJ.?IIe,&HED3*OD OO4O 0G(J D)9EA V' 2#11HI1ED5ON( '5 s{B$K=5%K,A+K KK$JA6K9A K  KK,'$K=1K K K K K) %K,, K: 6K== L cURS5nUS:wa"[R"[S5U-5e[R "SURS55up#[R "SURS55SnURU5nUR S5(d[R"[S 55e[URS5RS 55nX#U4$) NrNrs<only uncompressed stream clone bundles are supported; got %srrrrs@malformed stream clone bundle: requirements not properly encodedr,) rrrYr runpackendswithrrstripr=)rirrlrm requireslenrrs r$readbundle1headerr s''!*Kekk %     "===I--rwwqz215Kww{#H   U # #kk 5   xu-33D9:L  --r&c`[U5(a[R"[S55e[ U5up#nX@R - nU(a:[R"[S5SR [U55-5e[XX#5 [R"U5 g)zApply the content from a stream clone bundle version 1. We assume the 4 byte header has been read and validated and the file handle is at the 2 byte compression identifier. s2cannot apply stream clone bundle on non-empty repo4unable to apply stream clone: unsupported format: %sr-N) r1rrYr rr>r?r@r]rrc)r(rirlrmrrHs r$ applybundlev1r)s  4yykk C D  *;2)>&I,/Kkk E Fjj ,- .  d - %r&c,\rSrSrSrSSjrSSjrSrg)streamcloneapplieri@zClass to manage applying streaming clone bundles. We need to wrap ``applybundlev1()`` in a dedicated type to enable bundle readers to perform bundle type-specific functionality. cXlgN_fh)selffhs r$__init__streamcloneapplier.__init__Gsr&c,[XR5$r)rr)rr(s r$applystreamcloneapplier.applyJsT88,,r&rNrNone)__name__ __module__ __qualname____firstlineno____doc__rr__static_attributes__r&r$rr@s -r&rsccV/nUR5(dURS5 U$)z!list snapshot file from the stores phaseroots) publishingr)r(fnamess r$_walkstreamfullstorefilesrXs% F ??   m$ Mr&cZUup4pVU[:waU$X4XQ"X#RU554$)z actually copy the snapshot files) _filefullr?)rcopyvfsmapsrcrftypedatas r$ _filterfullr`s8"Cu  ud6;#3#3D#9: ;;r&c\rSrSrSr\R (aSOSrSSjrSr Sr SSjr SS jr SS jr SS jrSS jr\R"S 5rSrg)VolatileManagerihaManage temporary backups of volatile files during stream clone. This class will keep open file handles for the volatile files, writing the smaller ones on disk if the number of open file handles grow too much. This should be used as a Python context, the file handles and copies will be discarded when exiting the context. The preservation can be done by calling the object on the real path (encoded full path). Valid filehandles for any file should be retrieved by calling `open(path)`. rNdc<SUlSUlSUlSUlgNr)_counter _volatile_fps_copies_dst_dirrs r$rVolatileManager.__init__}s !  r&c|URS:XaURbe0UlU=RS- slU$Nrr )rr r s r$ __enter__VolatileManager.__enter__s; ==A %%- --!#D    r&cU=RS-slURS:XaURR5Hup4UR5 M SUlURbWURR5Hn[ R "U5 M [ R"UR5 SUlSUlURbeURbeURbegg)zdiscard all backupsr rN) rr valuescloser r tryunlinktryrmdirr )rargskwars_sizeritmps r$__exit__VolatileManager.__exit__s  ==A !//668  9!%D ||'<<..0CNN3'1 dmm,DL DM%%- --<<' ''==( (( r&cSUR:deURbeURbe0Ul[R"SS9Ulg)zpprepare a temporary directory to save volatile files This will be used as backup if we have too many files openrNs hg-clone-)prefix)rr r rmkdtempr s r$_init_tmp_copies VolatileManager._init_tmp_copiessO4==   ||###}}$$$  (( = r&clURcUR5 URS-n[URR 55SUHunup4[ RRU5n[R"XPRS9upgXpRU'[ R"U5 [US5nURS5 URUR!55 SSS5 URU UR5 M g!,(df  N0=f)z4move some of the open files to tempory files on diskNrN)rdirwbr)r r MAX_OPENr@r itemsosrbasenamermkstempr ropenseekrr) r flush_countrrrirfddstbcks r$_flush_some_on_disk#VolatileManager._flush_some_on_disks <<   ! ! #mmq( %d&8&8&>&>&@A,;OOC$WW%%c*F&&f--HGB #LL  HHRLc4C  "'')$!""3' HHJ P!s 1D%% D3 cS[US54=up#URU'URS[R5 UR 5nXC4URU'U$)z-preserve an open file handle for a given pathNrbr)r*r r+r'SEEK_ENDtell)rrr rirs r$ _keep_oneVolatileManager._keep_onesZ,0c4*AA""3' 2;;wwy#'*3 r&cSUR:de[UR5URS- :aUR 5 UR U5 g)z!preserve the volatile file at srcrr N)rr1r r%r0r6)rrs r$__call__VolatileManager.__call__sH4==   t!! "t}}q'8 9  $ $ & sr&cBSUR:de[UR5URS- :aUR 5 UR U5$![ a9nUR[R[R4;aeSnAgSnAff=f)zrecord a volatile file and returns it size return None if the file does not exists. Used for cache file that are not lock protected. rr N) rr1r r%r0r6OSErrorerrnoENOENTEPERM)rrerrs r$try_keepVolatileManager.try_keeps4==   t!! "t}}q'8 9  $ $ & >>#& & yyu{{ ;; s A B%/BBc#J# SUR:deURRU5nUbUup4URS5 Uv gURcUnOURRX5n[ US5nUv SSS5 g!,(df  g=f7f)Nrr3)rr r6r+r r*)rrrrri actual_paths r$r*VolatileManager.opens4==   ""&&s+  IE GGAJH||#! "ll..s8 k4(B)((sBB#B B# B B#)r rr r Nr)rbytesrrV)rrFrr)rrFrz Optional[int])rrrrrr iswindowsr%rrrr r0r6r9rA contextlibcontextmanagerr*rrr&r$rrhsZ $&&qCH )">&"  r&rc[UR[UR0nURUR 5;deU$)z$make a (src -> vfs) map for the repo) _srcstorer` _srccachecachevfsrr)r(rs r$_makemaprNs< 4994==F 886==? ** * Mr&c ## [U5nURUR5;a[R"S5eUVVs/sH up4X2UU4PM nnn[ U5 [ U5nS=pg[5n[R"5 UHFupnURX5 UR5Hn US- nX{RU 5- nM MH SSS5 URR[S5U[S5S9n U R!S5 U  Xg4v Sn UHupnUR#UU UUSS 9nUHunnnUv [R$"[ U55v [R$"U5v Uv SnUH4nU[ U5- nU [ U5- n U R!U 5 Uv M6 UU:wdM[S 5n[R&"UUUU4-5e M SSS5 SSS5 gs snnf!,(df  GND=f!,(df  N/=f!,(df  g=f7f) zactually emit the stream bundle9repo.vfs must not be added to vfsmap for security reasonsrr NrrrT)r(r volatiles max_changesetpreserve_file_counts=clone could only read %d bytes from %s, but expected %d bytes)rNrrrProgrammingErrorrr1rrnogcpreserve_volatilesrrr;rr r! get_streams uvarintencoderY)r(rrvfs_keyrh max_linkrevr totalfilesizerQkrrrtotalbytecountr entry_streamsrstreamrrmrmsgs r$_emit2ras+ d^F  xx6==?"$$ G  BII'!,GId#d)K!""J  i YY[$ $$S4A!OJ![[%55M#%77'' iL AhK(   + +N& ! ! '"-(, !.! +8&D&$I,,SY77,,T22J !I!'!SZ/ &#e*4 7# "( !D( 1$kk#D$0G*GHH%+8 '    J[X  scAIH&IIA H #AI+B-H2-H2 II H/ *I2 I <I I Ic ## [U5nURUR5;a[R"S5e[ U5n[ U5 /n[5nUHupVX%nSnUR(aNSnURXt5 UR5H'n U R(aU RU5 SnM) U(dMsURXRUU45 M [U5n [U5n URR![#S5U [#S5S9n U R%S5 Sv U  [&R("U 5v UHupnUR+UUUU S 9n[&R("[U55v UHRunnnU v [&R("[U55v [&R("U5v Uv UShvN MT U R-5 M SSS5 SSS5 gN-!,(df  N=f!,(df  g=f7f) z$actually emit the stream bundle (v3)rPTFrsentryrrN)r(rrQrR)rNrrrrTlistrrmaybe_volatilerVr is_volatilerrr1r;rr r!rrXrWr)r(rr base_entriesrQrYrhr any_filesrtotal_entry_countrZrrr^rr_rs r$_emit3ri:s d^F  xx6==?"$$ G  =Ld#G  i&JG/CI! $$S4A}} C( $I # y!<=' L$i 77'' iL#8(   $$%67 7& ! ! '"- !.! ((]);<<*7&D&$I,,SY77,,T22J%%% +8 ""$ '7  V&!X7  sQA"I$A2H2A4H2B#H!1H 2H!H2 IH!! H/ +H22 I<Icg)za function for synchronisation during tests Triggered after gather entry, but before starting to process/preserve them under lock. (on v1 is triggered before the actual walk start) Nrr(s r$rr{r&cg)za function for synchronisation during tests Triggered right before releasing the lock, but after computing what need needed to compute under lock. Nrrks r$rrrlr&cg)zWa function for synchronisation during tests Triggered right after releasing the lock. Nrrks r$rrrlr&cJ^\rSrSrSrSU4SjjrSSjrSU4SjjrSrU=r $) CacheEntryizRepresent an entry for Cache files It has special logic to preserve cache file early and accept optional presence. (Yes... this is not really a StoreEntry, but close enough. We could have a BaseEntry base class, bbut the store one would be identical) c">[TU]USS9 g)NT)re)superr)r entry_path __class__s r$rCacheEntry.__init__s    r&cURURUR55UlURc/UlgUR (de[ URURUR S9/Ulg)N)rrre)rAr? _entry_path _file_size_files _is_volatile CacheFile)rrrQs r$rVCacheEntry.preserve_volatilessm#,,SXXd6F6F-GH ?? "DK$$ $$#'#3#3"oo $ 1 1DKr&c>URc$[URURS9/Ul[TU]5$)N)rre)ryr{rwrzrrr)rrts r$rCacheEntry.filess> ;; #'#3#3 $ 1 1DK w}r&)rxryr)rzlist[store.StoreFile]) rrrrrrrVrr __classcell__rts@r$rprps  r&rpc$\rSrSr%SrS\S'Srg)r{iTr8optionalrN)rrrrr__annotations__rrr&r$r{r{sHdr&r{c## URUR5ceSnU(dU(a![R"URX5nUR 5(+n[ R"5 [UUUUS9nUH n[U4v M [R"U5H6nURRU5(dM%[[US94v M8 SSS5 g!,(df  g=f7f)zemit a seris of files information useful to clone a repo return (vfs-key, entry) iterator Where `entry` is StoreEntry. (used even for cache entries) Nrw)rs) _currentlock_lockrefrmatchrootrrrUr|rKr cachetocopyrMexistsrLrp) r(includesexcludesincludeobsmarkersr{rxrrrs r$ _entries_walkrs   T]] + 77 7G8""499hA! !E "  *   Ee$ $))$/D}}##D)) *"=>>0 s%A:Dn[8R:"[DU4S9nUR?U5 URA5 M@ URK5 UbURO[QU55 UHnURS5 M S S S 5 TRUSS9 S S S 5 URW5 URYS5nU(ae[[TXRUR\5 S S S 5 g ! UbURM5 e=f!UbURO[QU55 UHnURS5 M f=f!,(df  N=f!,(df  N=f!,(df  g =f)zApply the contents from a version 2 streaming clone. Data is read from an object that only needs to provide a ``read(size)`` method. rrrrrPscpusmemorysworkers!parallel-stream-bundle-processings,parallel-stream-bundle-processing.num-writerrs/parallel-stream-bundle-processing.memory-targetNc3X># UHoRTR5v M! g7frrr;.0rr(s r$ consumev2..z!Nos))$''22o'*) memory_target)targetr) file_chunker mark_usedTrr )/r\r;rTr rrmrr~rrNrrrrTrget_resource_profilerr configintDEFAULT_NUM_WRITER configbytesDEFAULT_MEMORY_TARGETrr _FileChunker_ThreadSafeFileChunker _DataQueuer chunkbuffer threadingThread fill_fromrr_v2_parse_files _write_files_FileInfoQueuerfillabortrr1r?rfrrrr)r(rirlfilesizerrreportr cpu_profile mem_profilethreaded num_writerrrworkers info_queue data_queuerfcraw_datawr__remainss` r$ consumev2rHs   3 4$..23 4 77'' hK8(   $ 88v}} &((K 22477FC 22477IF 77%% ; WW&&  ;  ?+K8J++  >  1  M a 1+>M   h 'NfmmoND! !  3!#)#%3%/m%L (4(2(<(A(N&N '6OO,5A O=N N  8O O O) %O,, O: 6O== P c*\rSrSrSrSrSrSrSrg)_QueueizDa reimplementation of queue.Queue which doesn't use thread.Conditionc[R"5Ul[R"5Ul[R"5Ulgr) collectionsdeque_queuerLock_lock_waitr s r$r_Queue.__init__s0!'') ^^% ^^% r&cUR URRU5 URR 5(aURR 5 SSS5 g!,(df  g=fr)rrrrlockedrelease)ritems r$put _Queue.putsH ZZ KK  t $zz  "" ""$ ZZs AA++ A9c^UR [UR5S:XaURR SS9 URR 5 URR 5 URR 5 URR 5(a3[UR5S:aURR 5 [UR5S:XaMURR5sSSS5 $!,(df  g=f)NrFblockingr )rr1rracquirerrpopleftr s r$r6 _Queue.gets ZZdkk"a' ""E"2 ""$ ""$ ""$ ::$$&&3t{{+;a+?JJ&&(dkk"a' ;;&&(#ZZsC,D;D D,)rrrN) rrrrrrrr6rrr&r$rrsN &%)r&rc@\rSrSrSrS SjrSrSrSrSr S r S r g) rizA queue passing data from the bundle stream to other thread It has a "memory_target" optional parameter to avoid buffering too much information. The implementation is not exact and the memory target might be exceed for a time in some situation. Nc&[5UlSUlXlURb&URS::a[R "S5e[ R"5Ul[ R"5Ul SUl SUl g)NFrzmemory target should be > 0) r_q_abort_memory_targetrrTrrrr _current_read _current_used)rrs r$r_DataQueue.__init__sq( +    *t/B/Ba/G(()FG G^^% ^^% r&cjURcgURUR- UR:$)zfTrue if more data can be read without further exceeding memory target Must be called under the lock. T)rrrr s r$_has_free_space_DataQueue._has_free_spaces7    &""T%7%774;N;NNNr&c2URbyUR XR:aXlURR 5(a/UR 5(aURR 5 SSS5 gg!,(df  g=f)z{Notify we have used the buffer up to "offset" This is meant to be used from another thread than the one filler the queue. N)rrrrrrr)roffsets r$r_DataQueue.mark_used!sl    *...)/&::$$&&4+?+?+A+AJJ&&(  +s A$B Bc URnUGH6nU=R[U5- slURU5 UR(a OUR cMUUR  UR5(dURRSS9 UR R5 URR5 UR R5 URR5(aURR5 UR5(dMSSS5 GM9 URS5 g!,(df  GM]=f!URS5 f=f)zffill the data queue from a bundle2 part object This is meant to be called by the data reading thread NFr) rrr1rrrrrrrrr)rrqrs r$r_DataQueue.fill_from.s GG ""c$i/"d ;;&&2"&"6"6"8"8!JJ...> JJ..0 JJ..0 JJ..0 $zz0022 $ 2 2 4#'"6"6"8"8$ * EE$K$ EE$Ks+AE0% E01C E> E0 E- ' E00Fc## URnUR5=nb,Uv UR(agUR5=nbM+gg7f)zWIterate over the bundle chunkgs This is meant to be called by the data parsing thread.N)rr6rrrrs r$__iter___DataQueue.__iter__LsA GGEEG|q(G{{EEG|q(s A A Ac"SUlURRS5 URbOUR UR R 5(aUR R5 SSS5 gg!,(df  g=f)zjstop the data-reading thread and interrupt the comsuming iteration This is meant to be called on errors. TN)rrrrrrrrr s r$r_DataQueue.abortVsc    D    *::$$&&JJ&&( +s :B B)rrrrrrrr) rrrrrrrrrrrrrr&r$rrs&, O )< )r&rc4\rSrSrSrS SjrSrSrSrSr g) ridzDA thread-safe queue to passer parsed file information to the writersc.Xl[5Ulgr)_inforr)rinfos r$r_FileInfoQueue.__init__gs (r&cdURnURHnURU5 M g)ziterate over the parsed information to file the queue This is meant to be call from the thread parsing the stream information. N)rrrrs r$r_FileInfoQueue.fillks& GGA EE!Hr&c^[U5HnURRS5 M g)zsignal all the workers that we no longer have any file info coming Called from the thread parsing the stream information (and/or the main thread if different). N)rrr)r number_workerrs r$r_FileInfoQueue.closets#  &B GGKK 'r&c#~# URnUR5=nbUv UR5=nbMgg7f)zZiterate over the available file info This is meant to be called from the writer threads. N)rr6rs r$r_FileInfoQueue.__iter__}s7 GGEEG|q(GEEG|q(s7==)rrNrzIterable[FileInfoT]) rrrrrrrrrrrr&r$rrdsNr&rcN\rSrSrSrSS SjjrS SjrS SjrSrg) rizyield the chunk that constitute a file This class exists as the counterpart of the threaded version and would not be very useful on its own. NcNX@lX0l[R"XS9Ulg)Nr)rrrr_chunks)rridata_lenrrrs r$r_FileChunker.__init__s   ))"= r&cg)z#Do nothing in non-threading contextNrr s r$r_FileChunker.fillrlr&c## URHQnUR=R[U5- slURR [U5S9 Uv MS g7f)Nr)r rrr1rr)rrs r$r_FileChunker.__iter__sJ\\E KK " "c%j 0 " MM # #U # 4K"sA#A%)r rrr ribundle2mod.unbundlepartr rVrscmutil.progressrrrOptional[Callable[[int], None]]rr FileChunksT) rrrrrrrrrrr&r$rrsK6: > # > ># >  > 3 >2r&rcb^\rSrSrSrSSU4SjjjrS U4SjjrS SjrSrU=r $) rizyield the chunk that constitute a file Make sure you call the "fill" function in the main thread to read the right data at the right time. c\>[TU]XX45 Xl[5UlXPlgr)rrr_fprr _mark_used)rrir rrrrts r$r_ThreadSafeFileChunker.__init__s' x8h #r&c>[TU]5H9nURR5nURR X45 M; URR S5 g!URR S5 f=f)zfill the file chunker queue with data read from the stream This is meant to be called from the thread parsing information (and consuming the stream data). N)rrrrr5rr)rrrrts r$r_ThreadSafeFileChunker.fills^  ")+ 0, KKOOD !DKKOOD !s A A**Bc## URR5=nbFUup#URbURU5 Uv URR5=nbMEgg7f)zUIterate over all the file chunk This is meant to be called from the writer threads. N)rr6r)rrrrs r$r_ThreadSafeFileChunker.__iter__sW {{((t5 ME*'K {{((t5s A!A'%A')rrrrrrr) rrrrrrrrrrrs@r$rrsV6: $ # $ $# $  $ 3 $ $ "  r&rc#*# UbU"U5 Uv g7f)zused for single chunk file,Nr)rrrs r$ _trivial_filer"s & Ksc #T# [5nURS5 [U5GHyn [R"US5n [R "U5n [R "U5n [R"X5n UR R(a4UR RSX[R"U 54-5 X*nURX5unnU [R::awURU 5nUR5nU=R[U5- slUR![U5S9 [#UUU5nUU[%U54v GMOU"UU UUUS9nUU[%U54v UR'5 GM| g7f)zsdo the "stream-parsing" part of stream v2 The parsed information are yield result for consumption by the "writer" rr adding [%s] %s (%s) r)rN)rr!rr readexactlyuvarintdecodestreamr;rrrmprepare_streamed_fileDEFAULT_FILE_CHUNKrr5rr1rr"iterr)r(rivfs_maprrrrr known_dirsrrnamelendatalenrrrmodecrrs r$rrs`J OOA : r1%**2.**2., 77   GGMM(Ct~~g7N+OO l..t@ d d-- - AWWYF   Q '    CF  +"1i8FtF|, ,!# FtF|, , KKM=sF&F(c$[R[R-n[R(aU[R -nUHup#nUc[R "X!5nO[R "X!US9nUHTn[R"XV5nU[U5:dM*[R"XVUS5nU[U5:aM*MV [R"U5 M g![R"U5 f=f)zwrite files from parsed dataN)r.) r'O_WRONLYO_CREATrrGO_BINARYr*rr1r)rio_flagsrr.rr-rwrittens r$rr s{{RZZ'HBKK D <(Bd3B ((2-E * hhr?;GE * HHRL! HHRLs(C7.(C7C77Dc L^TR5 [R"5n[R"U5nTRR [ S5U-5 TRR[ S5U[ S5S9nURS5 Sn[T5nTRUR5;a[R"S5eSnTRS5 U4SjUR55n[U6 [!U5GHpn [R"U5n U S:Xa9TRR"(aTRR%SU -5 Xz- n[!U 5Hn [R&"US 5n Xkn [R"U5n [R"U5n[R&"X5nTRR"(a8S nUX[R("U54-nTRR%U5 X^- nU "US 5n[R*"XS 9HnUR-U5 M S S S 5 M UR/S S9 GMs S S S 5 TR1SS9 S S S 5 UR35 [5TX'U5 S S S 5 g !,(df  GMb=f!,(df  N^=f!,(df  NX=f!,(df  g =f)zApply the contents from a version 3 streaming clone. Data is read from an object that only needs to provide a ``read(size)`` method. s%d entries to transfer rsentriesrrrPc3X># UHoRTR5v M! g7frrrs r$rconsumev3..Arrsentry with no files [%d] r r$rrNrTr)r\rr~r&r;rTr rr!rNrrrrTrrrrrr%rmrrrrfrr)r(rir entrycountrbytes_transferredrrrrrlrrr,r-rr`rrs` r$ consumev3r;!sv  --b1  q45DE77'' hK:(  $ 88v}} &((K    h 'NfmmoNDz*A $ 8 8 "&":":2">#//<77,,":CCt~~g/F#GGC GGMM#.)4) t_)-););B)N # % 0*O-_."&&A&./+8 OO4O 0=(@ D%;LMu `-_+( '3 sUC%L7#LD4K3-K ; K3L2%L K0* K33 L =L L L L#cfSSKJn UVs/sHofUR;dMUPM nnU(a:[R"[ S5SR [U55-5e[R"5 [XX#5 SSS5 [URU5Ul URURURUR5UR l[$R&"U5 [(R*"U5 gs snf!,(df  N=fNr rKrr-)rPrLr>rrYr r?r@rrUrr%rr^r;r_r`rarrbrrc)r(rirlrrrLrrHs r$ applybundlev2r?ds*Flt~~.E1lKFkk E Fjj ,- .  $I0 6 D"88 ""DMMDII !!$' %%G sDD D"" D0cSSKJn UVs/sHoDUR;dMUPM nnU(a>[S5nUSR [ U55-n[ R"U5e[X5 [URU5Ul URURURUR5URl[ R""U5 [$R&"U5 gs snfr=)rPrLr>r r?r@rrYr;r%rr^r;r_r`rarrbrrc)r(rirrLr>rHr`s r$ applybundlev3rA~s*Flt~~.E1lKFGH uzz&-..kk# d5 D"88 ""DMMDII !!$' %!Gs DDc ^^ S/m U U4SjnUHupVnXnXn URU5n U RU5n [RRU 5n [RR U 5(d[ R "U 5 U RU5 [ R"U U T SUSS9 TR5 M T S$![a U(deN-f=f)NTc0>STS'[S5Tlg)NFrscopying)r topic)hardlinkrsr$ copy_used_copy_files..copy_useds :r&rF)rEno_hardlink_cbcheck_fs_hardlink) r?r'rdirnamerrmakedirs register_filecopyfileFileNotFoundErrorr) src_vfs_map dst_vfs_maprrrFr\rrsrc_vfsdst_vfssrc_pathdst_pathrJrEs ` @r$ _copy_filesrUsvH'%..<<%<<%''//(+ww~~g&& MM' "d#  MM!!("'   /%0 A; !  s*CC32C3c URVs1sHnU[R;dMUiM nnURVs1sHnU[R;dMUiM nnX4:XdeUR5 UR5 [R "U5nUR S5nSn[RRU5(aSn[USSSS9n[U5n[U5n [U5n [SU55U-n URR[!S5U [!S 5S 9n UV VVs/sH5upUR#5HnXR$UR&4PM M7 nnn n[)XUU 5nU(a>[R "U5nUR S5n[*R,"UU5 SSS5 W R/5 W(aS nOS nURR1UW -5 UR3S 5nUR4R7U5 SSS5 [2R8"URR:UR<5 SSS5 gs snfs snfs snnn f!,(df  N=f!,(df  Nn=f!,(df  g=f)zUcopy all content from one local repository to another This is useful for local clones bookmarksrr NTrc3Z# UH!n[USR55v M# g7f)r N)r1r)rrhs r$rlocal_copy..s"AAc!A$**,//s)+slinkingsfiles)rDrrslinked %d files scopied %d files s localclone)rrWORKING_DIR_REQUIREMENTSr\r bookmarksvfsr?r'rrrrcrNsumr;rr rrrrUrrMrrrrrcleanup_undo_filesr<r*)src_repo dest_repor>src_store_requirementsdest_store_requirements src_book_vfs srcbookmarksbm_countrrO dest_vfs_map total_filesrrYrhrrrE dst_book_vfs dstbookmarksr`trs r$ local_copyris &&&A O<< < &'''A O<< < ' " << <   ]]_%11(;L',,\:LHww~~l++#"& G 7mG"8,K#I.LAAAHLK{{// m!x[0H#*")JGA**AJJ7"8")  #;eXNH(55i@ +00>  lL9WX  &C&C # +,  " "= 1R OO ! !" %2 &&y||'8'8):K:KLq   PA_f2 1i  sgJJJJ<K  CJ*rzs8# "     ! Q 1 , [ 9 ) *   [|HV&CHL#L;?L ),:="77t8 %99'9xIFX.8&. - -    <D @IF>%B(''(V ?@/>-15--` vGtuo   SM  *)*)Zi)i)X  F:*\*Z .  $(415.. .  .  .%./..b(@NF&&(+&;J& &4&,DHMr&