관리-도구
편집 파일: tokenizer.cpython-33.pyc
� 7�Re�, c @ s d d l m Z m Z m Z y e Z Wn e k r: Yn Xd d l m Z d d l m Z d d l m Z d d l m Z m Z d d l m Z m Z m Z d d l m Z m Z d d l m Z d d l m Z d d l m Z e e � Z Gd d � d e � Z d S( i ( u absolute_importu divisionu unicode_literals( u dequei ( u spaceCharacters( u entities( u asciiLettersu asciiUpper2Lower( u digitsu hexDigitsu EOF( u tokenTypesu tagTokenTypes( u replacementCharacters( u HTMLInputStream( u Triec s� | Ee Z d Z d Z d� d� d� d� d� d� � f d d � Z d d � Z d d � Z d� d� d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d � Z d d! � Z d" d# � Z d$ d% � Z d& d' � Z d( d) � Z d* d+ � Z d, d- � Z d. d/ � Z d0 d1 � Z d2 d3 � Z d4 d5 � Z d6 d7 � Z! d8 d9 � Z" d: d; � Z# d< d= � Z$ d> d? � Z% d@ dA � Z& dB dC � Z' dD dE � Z( dF dG � Z) dH dI � Z* dJ dK � Z+ dL dM � Z, dN dO � Z- dP dQ � Z. dR dS � Z/ dT dU � Z0 dV dW � Z1 dX dY � Z2 dZ d[ � Z3 d\ d] � Z4 d^ d_ � Z5 d` da � Z6 db dc � Z7 dd de � Z8 df dg � Z9 dh di � Z: dj dk � Z; dl dm � Z< dn do � Z= dp dq � Z> dr ds � Z? dt du � Z@ dv dw � ZA dx dy � ZB dz d{ � ZC d| d} � ZD d~ d � ZE d� d� � ZF d� d� � ZG d� d� � ZH d� d� � ZI d� d� � ZJ d� d� � ZK d� d� � ZL d� d� � ZM d� d� � ZN d� d� � ZO � S(� u HTMLTokenizeru This class takes care of tokenizing HTML. * self.currentToken Holds the token that is currently being processed. * self.state Holds a reference to the method to be invoked... XXX * self.stream Points to HTMLInputStream object. c sz t | | | | � | _ | | _ | | _ | | _ d | _ g | _ | j | _ d | _ d | _ t t | � j � d S( NF( u HTMLInputStreamu streamu parseru lowercaseElementNameu lowercaseAttrNameu Falseu escapeFlagu lastFourCharsu dataStateu stateu escapeu Noneu currentTokenu superu HTMLTokenizeru __init__( u selfu streamu encodingu parseMetau useChardetu lowercaseElementNameu lowercaseAttrNameu parser( u __class__( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu __init__% s u HTMLTokenizer.__init__c c s} t g � | _ xg | j � rx x6 | j j rV i t d d 6| j j j d � d 6Vq! Wx | j rt | j j � VqZ Wq Wd S( u� This is where the magic happens. We do our usually processing through the states and when we have a token to return we yield the token which pauses processing until the next token is requested. u ParseErroru typei u dataN( u dequeu tokenQueueu stateu streamu errorsu tokenTypesu popu popleft( u self( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu __iter__9 s * u HTMLTokenizer.__iter__c 0 C s� t } d } | r! t } d } n g } | j j � } x8 | | k rp | t k rp | j | � | j j � } q9 Wt d j | � | � } | t k r� t | } | j j i t d d 6d d 6i | d 6d 6� n�d | k o� d k n s� | d k r3d } | j j i t d d 6d d 6i | d 6d 6� nsd | k oJd k n s d | k ofd k n s d | k o�d k n s d | k o�d k n s | t d d d d d d d d d d d d! d" d# d$ d% d& d' d( d) d* d+ d, d- d. d/ d0 d1 d2 d3 d4 d5 d6 d7 d g# � k rQ| j j i t d d 6d d 6i | d 6d 6� n y t | � } WnB t k r�| d8 } t d | d ?B� t d9 | d: @B� } Yn X| d; k r�| j j i t d d 6d< d 6� | j j | � n | S(= u� This function returns either U+FFFD or the character based on the decimal or hexadecimal representation. It also discards ";" if present. If not present self.tokenQueue.append({"type": tokenTypes["ParseError"]}) is invoked. i i u u ParseErroru typeu$ illegal-codepoint-for-numeric-entityu datau charAsIntu datavarsi � i�� i�� u �i i i i i i� i� i� i i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i i � i� u ;u numeric-entity-without-semicolon( u digitsu hexDigitsu streamu charu EOFu appendu intu joinu replacementCharactersu tokenQueueu tokenTypesu frozensetu chru ValueErroru unget( u selfu isHexu allowedu radixu charStacku cu charAsIntu charu v( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu consumeNumberEntityI s` + u! HTMLTokenizer.consumeNumberEntityc C s� d } | j j � g } | d t k s] | d t d d f k s] | d k rt | | d k rt | j j | d � n�| d d k rpd } | j | j j � � | d d k r� d } | j | j j � � n | r� | d t k s� | r"| d t k r"| j j | d � | j | � } qD| j j i t d d 6d d 6� | j j | j � � d d j | � } n�xF | d t k r�t j d j | � � s�Pn | j | j j � � qsWy2 t j d j | d d � � � } t | � } Wn t k rd } Yn X| d k r�| d d k rG| j j i t d d 6d d 6� n | d d k r�| r�| | t k s�| | t k s�| | d k r�| j j | j � � d d j | � } qDt | } | j j | j � � | d j | | d � � 7} nK | j j i t d d 6d d 6� | j j | j � � d d j | � } | rf| j d d d | 7<n= | t k r{d } n d } | j j i t | d 6| d 6� d S( Nu &i u <u #i u xu Xu ParseErroru typeu expected-numeric-entityu datau u ;u named-entity-without-semicolonu =u expected-named-entityu SpaceCharactersu CharactersFi����( u xu XTi����i����i����i����i����i����i����i����( u streamu charu spaceCharactersu EOFu Noneu ungetu Falseu appendu Trueu hexDigitsu digitsu consumeNumberEntityu tokenQueueu tokenTypesu popu joinu entitiesTrieu has_keys_with_prefixu longest_prefixu lenu KeyErroru asciiLettersu entitiesu currentToken( u selfu allowedCharu fromAttributeu outputu charStacku hexu entityNameu entityLengthu tokenType( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu consumeEntity� sf ) " u HTMLTokenizer.consumeEntityc C s | j d | d d � d S( uI This method replaces the need for "entityInAttributeValueState". u allowedCharu fromAttributeNT( u consumeEntityu True( u selfu allowedChar( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu processEntityInAttribute� s u& HTMLTokenizer.processEntityInAttributec C s� | j } | d t k r� | j r< | d j t � | d <n | d t d k r� | d r | j j i t d d 6d d 6� n | d r� | j j i t d d 6d d 6� q� q� n | j j | � | j | _ d S( u� This method is a generic handler for emitting the tags. It also sets the state to "data" because that's what's needed after a token has been emitted. u typeu nameu EndTagu datau ParseErroru attributes-in-end-tagu selfClosingu self-closing-flag-on-end-tagN( u currentTokenu tagTokenTypesu lowercaseElementNameu translateu asciiUpper2Loweru tokenTypesu tokenQueueu appendu dataStateu state( u selfu token( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu emitCurrentToken� s u HTMLTokenizer.emitCurrentTokenc C s( | j j � } | d k r* | j | _ n� | d k rE | j | _ n� | d k r� | j j i t d d 6d d 6� | j j i t d d 6d d 6� n� | t k r� d S| t k r� | j j i t d d 6| | j j t d � d 6� n8 | j j d � } | j j i t d d 6| | d 6� d S( Nu &u <u u ParseErroru typeu invalid-codepointu datau Charactersu SpaceCharactersFT( u &u <u ( u streamu charu entityDataStateu stateu tagOpenStateu tokenQueueu appendu tokenTypesu EOFu Falseu spaceCharactersu charsUntilu True( u selfu datau chars( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu dataState� s& !u HTMLTokenizer.dataStatec C s | j � | j | _ d S( NT( u consumeEntityu dataStateu stateu True( u self( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu entityDataState s u HTMLTokenizer.entityDataStatec C s( | j j � } | d k r* | j | _ n� | d k rE | j | _ n� | t k rU d S| d k r� | j j i t d d 6d d 6� | j j i t d d 6d d 6� n| | t k r� | j j i t d d 6| | j j t d � d 6� n8 | j j d � } | j j i t d d 6| | d 6� d S( Nu &u <u u ParseErroru typeu invalid-codepointu datau Charactersu �u SpaceCharactersFT( u &u <u ( u streamu charu characterReferenceInRcdatau stateu rcdataLessThanSignStateu EOFu Falseu tokenQueueu appendu tokenTypesu spaceCharactersu charsUntilu True( u selfu datau chars( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu rcdataState s& !u HTMLTokenizer.rcdataStatec C s | j � | j | _ d S( NT( u consumeEntityu rcdataStateu stateu True( u self( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu characterReferenceInRcdata: s u( HTMLTokenizer.characterReferenceInRcdatac C s� | j j � } | d k r* | j | _ n� | d k r} | j j i t d d 6d d 6� | j j i t d d 6d d 6� nH | t k r� d S| j j d � } | j j i t d d 6| | d 6� d S( Nu <u u ParseErroru typeu invalid-codepointu datau Charactersu �F( u <u T( u streamu charu rawtextLessThanSignStateu stateu tokenQueueu appendu tokenTypesu EOFu Falseu charsUntilu True( u selfu datau chars( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu rawtextState? s u HTMLTokenizer.rawtextStatec C s� | j j � } | d k r* | j | _ n� | d k r} | j j i t d d 6d d 6� | j j i t d d 6d d 6� nH | t k r� d S| j j d � } | j j i t d d 6| | d 6� d S( Nu <u u ParseErroru typeu invalid-codepointu datau Charactersu �F( u <u T( u streamu charu scriptDataLessThanSignStateu stateu tokenQueueu appendu tokenTypesu EOFu Falseu charsUntilu True( u selfu datau chars( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataStateQ s u HTMLTokenizer.scriptDataStatec C s� | j j � } | t k r d S| d k rr | j j i t d d 6d d 6� | j j i t d d 6d d 6� n2 | j j i t d d 6| | j j d � d 6� d S( Nu u ParseErroru typeu invalid-codepointu datau Charactersu �FT( u streamu charu EOFu Falseu tokenQueueu appendu tokenTypesu charsUntilu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu plaintextStatec s u HTMLTokenizer.plaintextStatec C s� | j j � } | d k r* | j | _ nr| d k rE | j | _ nW| t k r� i t d d 6| d 6g d 6d d 6d d 6| _ | j | _ n | d k r� | j j i t d d 6d d 6� | j j i t d d 6d d 6� | j | _ n� | d k r<| j j i t d d 6d d 6� | j j | � | j | _ n` | j j i t d d 6d d 6� | j j i t d d 6d d 6� | j j | � | j | _ d S( Nu !u /u StartTagu typeu nameu datau selfClosingu selfClosingAcknowledgedu >u ParseErroru' expected-tag-name-but-got-right-bracketu Charactersu <>u ?u' expected-tag-name-but-got-question-marku expected-tag-nameu <FT( u streamu charu markupDeclarationOpenStateu stateu closeTagOpenStateu asciiLettersu tokenTypesu Falseu currentTokenu tagNameStateu tokenQueueu appendu dataStateu ungetu bogusCommentStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu tagOpenStater s6 " " u HTMLTokenizer.tagOpenStatec C s? | j j � } | t k rS i t d d 6| d 6g d 6d d 6| _ | j | _ n� | d k r� | j j i t d d 6d d 6� | j | _ n� | t k r� | j j i t d d 6d d 6� | j j i t d d 6d d 6� | j | _ nL | j j i t d d 6d d 6i | d 6d 6� | j j | � | j | _ d S( Nu EndTagu typeu nameu datau selfClosingu >u ParseErroru* expected-closing-tag-but-got-right-bracketu expected-closing-tag-but-got-eofu Charactersu </u! expected-closing-tag-but-got-charu datavarsFT( u streamu charu asciiLettersu tokenTypesu Falseu currentTokenu tagNameStateu stateu tokenQueueu appendu dataStateu EOFu ungetu bogusCommentStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu closeTagOpenState� s( " u HTMLTokenizer.closeTagOpenStatec C s� | j j � } | t k r* | j | _ n� | d k rC | j � n� | t k r� | j j i t d d 6d d 6� | j | _ nn | d k r� | j | _ nS | d k r� | j j i t d d 6d d 6� | j d d 7<n | j d | 7<d S( Nu >u ParseErroru typeu eof-in-tag-nameu datau /u u invalid-codepointu nameu �T( u streamu charu spaceCharactersu beforeAttributeNameStateu stateu emitCurrentTokenu EOFu tokenQueueu appendu tokenTypesu dataStateu selfClosingStartTagStateu currentTokenu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu tagNameState� s" u HTMLTokenizer.tagNameStatec C su | j j � } | d k r3 d | _ | j | _ n> | j j i t d d 6d d 6� | j j | � | j | _ d S( Nu /u u Charactersu typeu <u dataT( u streamu charu temporaryBufferu rcdataEndTagOpenStateu stateu tokenQueueu appendu tokenTypesu ungetu rcdataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu rcdataLessThanSignState� s " u% HTMLTokenizer.rcdataLessThanSignStatec C s{ | j j � } | t k r9 | j | 7_ | j | _ n> | j j i t d d 6d d 6� | j j | � | j | _ d S( Nu Charactersu typeu </u dataT( u streamu charu asciiLettersu temporaryBufferu rcdataEndTagNameStateu stateu tokenQueueu appendu tokenTypesu ungetu rcdataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu rcdataEndTagOpenState� s " u# HTMLTokenizer.rcdataEndTagOpenStatec C s� | j o( | j d j � | j j � k } | j j � } | t k r� | r� i t d d 6| j d 6g d 6d d 6| _ | j | _ n| d k r� | r� i t d d 6| j d 6g d 6d d 6| _ | j | _ n� | d k r+| r+i t d d 6| j d 6g d 6d d 6| _ | j � | j | _ nc | t k rI| j | 7_ nE | j j i t d d 6d | j d 6� | j j | � | j | _ d S( Nu nameu EndTagu typeu datau selfClosingu /u >u Charactersu </FT( u currentTokenu loweru temporaryBufferu streamu charu spaceCharactersu tokenTypesu Falseu beforeAttributeNameStateu stateu selfClosingStartTagStateu emitCurrentTokenu dataStateu asciiLettersu tokenQueueu appendu ungetu rcdataStateu True( u selfu appropriateu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu rcdataEndTagNameState� s2 + u# HTMLTokenizer.rcdataEndTagNameStatec C su | j j � } | d k r3 d | _ | j | _ n> | j j i t d d 6d d 6� | j j | � | j | _ d S( Nu /u u Charactersu typeu <u dataT( u streamu charu temporaryBufferu rawtextEndTagOpenStateu stateu tokenQueueu appendu tokenTypesu ungetu rawtextStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu rawtextLessThanSignState� s " u& HTMLTokenizer.rawtextLessThanSignStatec C s{ | j j � } | t k r9 | j | 7_ | j | _ n> | j j i t d d 6d d 6� | j j | � | j | _ d S( Nu Charactersu typeu </u dataT( u streamu charu asciiLettersu temporaryBufferu rawtextEndTagNameStateu stateu tokenQueueu appendu tokenTypesu ungetu rawtextStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu rawtextEndTagOpenState s " u$ HTMLTokenizer.rawtextEndTagOpenStatec C s� | j o( | j d j � | j j � k } | j j � } | t k r� | r� i t d d 6| j d 6g d 6d d 6| _ | j | _ n| d k r� | r� i t d d 6| j d 6g d 6d d 6| _ | j | _ n� | d k r+| r+i t d d 6| j d 6g d 6d d 6| _ | j � | j | _ nc | t k rI| j | 7_ nE | j j i t d d 6d | j d 6� | j j | � | j | _ d S( Nu nameu EndTagu typeu datau selfClosingu /u >u Charactersu </FT( u currentTokenu loweru temporaryBufferu streamu charu spaceCharactersu tokenTypesu Falseu beforeAttributeNameStateu stateu selfClosingStartTagStateu emitCurrentTokenu dataStateu asciiLettersu tokenQueueu appendu ungetu rawtextStateu True( u selfu appropriateu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu rawtextEndTagNameState s2 + u$ HTMLTokenizer.rawtextEndTagNameStatec C s� | j j � } | d k r3 d | _ | j | _ n{ | d k rp | j j i t d d 6d d 6� | j | _ n> | j j i t d d 6d d 6� | j j | � | j | _ d S( Nu /u u !u Charactersu typeu <!u datau <T( u streamu charu temporaryBufferu scriptDataEndTagOpenStateu stateu tokenQueueu appendu tokenTypesu scriptDataEscapeStartStateu ungetu scriptDataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataLessThanSignState' s "" u) HTMLTokenizer.scriptDataLessThanSignStatec C s{ | j j � } | t k r9 | j | 7_ | j | _ n> | j j i t d d 6d d 6� | j j | � | j | _ d S( Nu Charactersu typeu </u dataT( u streamu charu asciiLettersu temporaryBufferu scriptDataEndTagNameStateu stateu tokenQueueu appendu tokenTypesu ungetu scriptDataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataEndTagOpenState5 s " u' HTMLTokenizer.scriptDataEndTagOpenStatec C s� | j o( | j d j � | j j � k } | j j � } | t k r� | r� i t d d 6| j d 6g d 6d d 6| _ | j | _ n| d k r� | r� i t d d 6| j d 6g d 6d d 6| _ | j | _ n� | d k r+| r+i t d d 6| j d 6g d 6d d 6| _ | j � | j | _ nc | t k rI| j | 7_ nE | j j i t d d 6d | j d 6� | j j | � | j | _ d S( Nu nameu EndTagu typeu datau selfClosingu /u >u Charactersu </FT( u currentTokenu loweru temporaryBufferu streamu charu spaceCharactersu tokenTypesu Falseu beforeAttributeNameStateu stateu selfClosingStartTagStateu emitCurrentTokenu dataStateu asciiLettersu tokenQueueu appendu ungetu scriptDataStateu True( u selfu appropriateu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataEndTagNameState@ s2 + u' HTMLTokenizer.scriptDataEndTagNameStatec C sl | j j � } | d k rL | j j i t d d 6d d 6� | j | _ n | j j | � | j | _ d S( Nu -u Charactersu typeu dataT( u streamu charu tokenQueueu appendu tokenTypesu scriptDataEscapeStartDashStateu stateu ungetu scriptDataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataEscapeStartState\ s " u( HTMLTokenizer.scriptDataEscapeStartStatec C sl | j j � } | d k rL | j j i t d d 6d d 6� | j | _ n | j j | � | j | _ d S( Nu -u Charactersu typeu dataT( u streamu charu tokenQueueu appendu tokenTypesu scriptDataEscapedDashDashStateu stateu ungetu scriptDataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataEscapeStartDashStatef s " u, HTMLTokenizer.scriptDataEscapeStartDashStatec C s | j j � } | d k rL | j j i t d d 6d d 6� | j | _ n� | d k rg | j | _ n� | d k r� | j j i t d d 6d d 6� | j j i t d d 6d d 6� nS | t k r� | j | _ n8 | j j d � } | j j i t d d 6| | d 6� d S( Nu -u Charactersu typeu datau <u u ParseErroru invalid-codepointu �( u <u -u T( u streamu charu tokenQueueu appendu tokenTypesu scriptDataEscapedDashStateu stateu" scriptDataEscapedLessThanSignStateu EOFu dataStateu charsUntilu True( u selfu datau chars( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataEscapedStatep s" " u$ HTMLTokenizer.scriptDataEscapedStatec C s | j j � } | d k rL | j j i t d d 6d d 6� | j | _ n� | d k rg | j | _ n� | d k r� | j j i t d d 6d d 6� | j j i t d d 6d d 6� | j | _ nI | t k r� | j | _ n. | j j i t d d 6| d 6� | j | _ d S( Nu -u Charactersu typeu datau <u u ParseErroru invalid-codepointu �T( u streamu charu tokenQueueu appendu tokenTypesu scriptDataEscapedDashDashStateu stateu" scriptDataEscapedLessThanSignStateu scriptDataEscapedStateu EOFu dataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataEscapedDashState� s" " " u( HTMLTokenizer.scriptDataEscapedDashStatec C sD | j j � } | d k r@ | j j i t d d 6d d 6� n | d k r[ | j | _ n� | d k r� | j j i t d d 6d d 6� | j | _ n� | d k r� | j j i t d d 6d d 6� | j j i t d d 6d d 6� | j | _ nI | t k r| j | _ n. | j j i t d d 6| d 6� | j | _ d S( Nu -u Charactersu typeu datau <u >u u ParseErroru invalid-codepointu �T( u streamu charu tokenQueueu appendu tokenTypesu" scriptDataEscapedLessThanSignStateu stateu scriptDataStateu scriptDataEscapedStateu EOFu dataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataEscapedDashDashState� s& % " " u, HTMLTokenizer.scriptDataEscapedDashDashStatec C s� | j j � } | d k r3 d | _ | j | _ n� | t k r} | j j i t d d 6d | d 6� | | _ | j | _ n> | j j i t d d 6d d 6� | j j | � | j | _ d S( Nu /u u Charactersu typeu <u dataT( u streamu charu temporaryBufferu scriptDataEscapedEndTagOpenStateu stateu asciiLettersu tokenQueueu appendu tokenTypesu scriptDataDoubleEscapeStartStateu ungetu scriptDataEscapedStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu" scriptDataEscapedLessThanSignState� s & " u0 HTMLTokenizer.scriptDataEscapedLessThanSignStatec C su | j j � } | t k r3 | | _ | j | _ n> | j j i t d d 6d d 6� | j j | � | j | _ d S( Nu Charactersu typeu </u dataT( u streamu charu asciiLettersu temporaryBufferu scriptDataEscapedEndTagNameStateu stateu tokenQueueu appendu tokenTypesu ungetu scriptDataEscapedStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataEscapedEndTagOpenState� s " u. HTMLTokenizer.scriptDataEscapedEndTagOpenStatec C s� | j o( | j d j � | j j � k } | j j � } | t k r� | r� i t d d 6| j d 6g d 6d d 6| _ | j | _ n| d k r� | r� i t d d 6| j d 6g d 6d d 6| _ | j | _ n� | d k r+| r+i t d d 6| j d 6g d 6d d 6| _ | j � | j | _ nc | t k rI| j | 7_ nE | j j i t d d 6d | j d 6� | j j | � | j | _ d S( Nu nameu EndTagu typeu datau selfClosingu /u >u Charactersu </FT( u currentTokenu loweru temporaryBufferu streamu charu spaceCharactersu tokenTypesu Falseu beforeAttributeNameStateu stateu selfClosingStartTagStateu emitCurrentTokenu dataStateu asciiLettersu tokenQueueu appendu ungetu scriptDataEscapedStateu True( u selfu appropriateu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataEscapedEndTagNameState� s2 + u. HTMLTokenizer.scriptDataEscapedEndTagNameStatec C s� | j j � } | t t d � Bk rz | j j i t d d 6| d 6� | j j � d k rk | j | _ q� | j | _ n\ | t k r� | j j i t d d 6| d 6� | j | 7_ n | j j | � | j | _ d S( Nu /u >u Charactersu typeu datau script( u /u >T( u streamu charu spaceCharactersu frozensetu tokenQueueu appendu tokenTypesu temporaryBufferu loweru scriptDataDoubleEscapedStateu stateu scriptDataEscapedStateu asciiLettersu ungetu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataDoubleEscapeStartState� s " " u. HTMLTokenizer.scriptDataDoubleEscapeStartStatec C s? | j j � } | d k rL | j j i t d d 6d d 6� | j | _ n� | d k r� | j j i t d d 6d d 6� | j | _ n� | d k r� | j j i t d d 6d d 6� | j j i t d d 6d d 6� n_ | t k r| j j i t d d 6d d 6� | j | _ n" | j j i t d d 6| d 6� d S( Nu -u Charactersu typeu datau <u u ParseErroru invalid-codepointu �u eof-in-script-in-scriptT( u streamu charu tokenQueueu appendu tokenTypesu scriptDataDoubleEscapedDashStateu stateu( scriptDataDoubleEscapedLessThanSignStateu EOFu dataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataDoubleEscapedState� s$ " " "u* HTMLTokenizer.scriptDataDoubleEscapedStatec C sW | j j � } | d k rL | j j i t d d 6d d 6� | j | _ n| d k r� | j j i t d d 6d d 6� | j | _ n� | d k r� | j j i t d d 6d d 6� | j j i t d d 6d d 6� | j | _ nk | t k r%| j j i t d d 6d d 6� | j | _ n. | j j i t d d 6| d 6� | j | _ d S( Nu -u Charactersu typeu datau <u u ParseErroru invalid-codepointu �u eof-in-script-in-scriptT( u streamu charu tokenQueueu appendu tokenTypesu$ scriptDataDoubleEscapedDashDashStateu stateu( scriptDataDoubleEscapedLessThanSignStateu scriptDataDoubleEscapedStateu EOFu dataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataDoubleEscapedDashState s( " " " u. HTMLTokenizer.scriptDataDoubleEscapedDashStatec C s� | j j � } | d k r@ | j j i t d d 6d d 6� nD| d k r} | j j i t d d 6d d 6� | j | _ n| d k r� | j j i t d d 6d d 6� | j | _ n� | d k r| j j i t d d 6d d 6� | j j i t d d 6d d 6� | j | _ nk | t k rV| j j i t d d 6d d 6� | j | _ n. | j j i t d d 6| d 6� | j | _ d S( Nu -u Charactersu typeu datau <u >u u ParseErroru invalid-codepointu �u eof-in-script-in-scriptT( u streamu charu tokenQueueu appendu tokenTypesu( scriptDataDoubleEscapedLessThanSignStateu stateu scriptDataStateu scriptDataDoubleEscapedStateu EOFu dataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu$ scriptDataDoubleEscapedDashDashState s, % " " " u2 HTMLTokenizer.scriptDataDoubleEscapedDashDashStatec C su | j j � } | d k rU | j j i t d d 6d d 6� d | _ | j | _ n | j j | � | j | _ d S( Nu /u Charactersu typeu datau T( u streamu charu tokenQueueu appendu tokenTypesu temporaryBufferu scriptDataDoubleEscapeEndStateu stateu ungetu scriptDataDoubleEscapedStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu( scriptDataDoubleEscapedLessThanSignState9 s " u6 HTMLTokenizer.scriptDataDoubleEscapedLessThanSignStatec C s� | j j � } | t t d � Bk rz | j j i t d d 6| d 6� | j j � d k rk | j | _ q� | j | _ n\ | t k r� | j j i t d d 6| d 6� | j | 7_ n | j j | � | j | _ d S( Nu /u >u Charactersu typeu datau script( u /u >T( u streamu charu spaceCharactersu frozensetu tokenQueueu appendu tokenTypesu temporaryBufferu loweru scriptDataEscapedStateu stateu scriptDataDoubleEscapedStateu asciiLettersu ungetu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu scriptDataDoubleEscapeEndStateD s " " u, HTMLTokenizer.scriptDataDoubleEscapeEndStatec C s� | j j � } | t k r1 | j j t d � nz| t k rf | j d j | d g � | j | _ nE| d k r | j � n,| d k r� | j | _ n| d k r� | j j i t d d 6d d 6� | j d j | d g � | j | _ n� | d k rH| j j i t d d 6d d 6� | j d j d d g � | j | _ nc | t k r�| j j i t d d 6d d 6� | j | _ n& | j d j | d g � | j | _ d S( Nu datau u >u /u 'u "u =u <u ParseErroru typeu# invalid-character-in-attribute-nameu u invalid-codepointu �u# expected-attribute-name-but-got-eofT( u 'u "u =u <( u streamu charu spaceCharactersu charsUntilu Trueu asciiLettersu currentTokenu appendu attributeNameStateu stateu emitCurrentTokenu selfClosingStartTagStateu tokenQueueu tokenTypesu EOFu dataState( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu beforeAttributeNameStateT s6 u& HTMLTokenizer.beforeAttributeNameStatec C s� | j j � } d } d } | d k r6 | j | _ n�| t k rw | j d d d | | j j t d � 7<d } nG| d k r� d } n2| t k r� | j | _ n| d k r� | j | _ n� | d k r| j j i t d d 6d d 6� | j d d d d 7<d } n� | d k rb| j j i t d d 6d d 6� | j d d d | 7<d } n\ | t k r�| j j i t d d 6d d 6� | j | _ n | j d d d | 7<d } | r|| j r�| j d d d j t � | j d d d <n xf | j d d d � D]M \ } } | j d d d | k r| j j i t d d 6d d 6� PqqW| r|| j � q|n d S( Nu =u datai i u >u /u u ParseErroru typeu invalid-codepointu �u 'u "u <u# invalid-character-in-attribute-nameu eof-in-attribute-nameu duplicate-attributeTFi����i����( u 'u "u <i����i����i����i����i����i����( u streamu charu Trueu Falseu beforeAttributeValueStateu stateu asciiLettersu currentTokenu charsUntilu spaceCharactersu afterAttributeNameStateu selfClosingStartTagStateu tokenQueueu appendu tokenTypesu EOFu dataStateu lowercaseAttrNameu translateu asciiUpper2Loweru emitCurrentToken( u selfu datau leavingThisStateu emitTokenu nameu value( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu attributeNameStater sT 0$ u HTMLTokenizer.attributeNameStatec C s� | j j � } | t k r1 | j j t d � n�| d k rL | j | _ nz| d k re | j � na| t k r� | j d j | d g � | j | _ n,| d k r� | j | _ n| d k r | j j i t d d 6d d 6� | j d j d d g � | j | _ n� | d k rc| j j i t d d 6d d 6� | j d j | d g � | j | _ nc | t k r�| j j i t d d 6d d 6� | j | _ n& | j d j | d g � | j | _ d S( Nu =u >u datau u /u u ParseErroru typeu invalid-codepointu �u 'u "u <u& invalid-character-after-attribute-nameu expected-end-of-tag-but-got-eofT( u 'u "u <( u streamu charu spaceCharactersu charsUntilu Trueu beforeAttributeValueStateu stateu emitCurrentTokenu asciiLettersu currentTokenu appendu attributeNameStateu selfClosingStartTagStateu tokenQueueu tokenTypesu EOFu dataState( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu afterAttributeNameState� s: u% HTMLTokenizer.afterAttributeNameStatec C s� | j j � } | t k r1 | j j t d � n�| d k rL | j | _ n�| d k rw | j | _ | j j | � nd| d k r� | j | _ nI| d k r� | j j i t d d 6d d 6� | j � n| d k r#| j j i t d d 6d d 6� | j d d d d 7<| j | _ n� | d k ry| j j i t d d 6d d 6� | j d d d | 7<| j | _ nb | t k r�| j j i t d d 6d d 6� | j | _ n% | j d d d | 7<| j | _ d S( Nu "u &u 'u >u ParseErroru typeu. expected-attribute-value-but-got-right-bracketu datau u invalid-codepointi u �u =u <u `u" equals-in-unquoted-attribute-valueu$ expected-attribute-value-but-got-eofTi����( u =u <u `i����i����( u streamu charu spaceCharactersu charsUntilu Trueu attributeValueDoubleQuotedStateu stateu attributeValueUnQuotedStateu ungetu attributeValueSingleQuotedStateu tokenQueueu appendu tokenTypesu emitCurrentTokenu currentTokenu EOFu dataState( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu beforeAttributeValueState� s> u' HTMLTokenizer.beforeAttributeValueStatec C s� | j j � } | d k r* | j | _ n� | d k rF | j d � n� | d k r� | j j i t d d 6d d 6� | j d d d d 7<nf | t k r� | j j i t d d 6d d 6� | j | _ n) | j d d d | | j j d � 7<d S( Nu "u &u u ParseErroru typeu invalid-codepointu datai u �u# eof-in-attribute-value-double-quotei����i����( u "u &u T( u streamu charu afterAttributeValueStateu stateu processEntityInAttributeu tokenQueueu appendu tokenTypesu currentTokenu EOFu dataStateu charsUntilu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu attributeValueDoubleQuotedState� s u- HTMLTokenizer.attributeValueDoubleQuotedStatec C s� | j j � } | d k r* | j | _ n� | d k rF | j d � n� | d k r� | j j i t d d 6d d 6� | j d d d d 7<nf | t k r� | j j i t d d 6d d 6� | j | _ n) | j d d d | | j j d � 7<d S( Nu 'u &u u ParseErroru typeu invalid-codepointu datai u �u# eof-in-attribute-value-single-quotei����i����( u 'u &u T( u streamu charu afterAttributeValueStateu stateu processEntityInAttributeu tokenQueueu appendu tokenTypesu currentTokenu EOFu dataStateu charsUntilu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu attributeValueSingleQuotedState� s u- HTMLTokenizer.attributeValueSingleQuotedStatec C sg | j j � } | t k r* | j | _ n9| d k rF | j d � n | d k r_ | j � n| d k r� | j j i t d d 6d d 6� | j d d d | 7<n� | d k r� | j j i t d d 6d d 6� | j d d d d 7<np | t k r0| j j i t d d 6d d 6� | j | _ n3 | j d d d | | j j t d � t B� 7<d S( Nu &u >u "u 'u =u <u `u ParseErroru typeu0 unexpected-character-in-unquoted-attribute-valueu datai u u invalid-codepointu �u eof-in-attribute-value-no-quotes( u "u 'u =u <u `i����i����i����( u &u >u "u 'u =u <u `u T( u streamu charu spaceCharactersu beforeAttributeNameStateu stateu processEntityInAttributeu emitCurrentTokenu tokenQueueu appendu tokenTypesu currentTokenu EOFu dataStateu charsUntilu frozensetu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu attributeValueUnQuotedState s, u) HTMLTokenizer.attributeValueUnQuotedStatec C s� | j j � } | t k r* | j | _ n� | d k rC | j � n� | d k r^ | j | _ n� | t k r� | j j i t d d 6d d 6� | j j | � | j | _ n> | j j i t d d 6d d 6� | j j | � | j | _ d S( Nu >u /u ParseErroru typeu$ unexpected-EOF-after-attribute-valueu datau* unexpected-character-after-attribute-valueT( u streamu charu spaceCharactersu beforeAttributeNameStateu stateu emitCurrentTokenu selfClosingStartTagStateu EOFu tokenQueueu appendu tokenTypesu ungetu dataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu afterAttributeValueState* s" u& HTMLTokenizer.afterAttributeValueStatec C s� | j j � } | d k r5 d | j d <| j � n� | t k r� | j j i t d d 6d d 6� | j j | � | j | _ n> | j j i t d d 6d d 6� | j j | � | j | _ d S( Nu >u selfClosingu ParseErroru typeu# unexpected-EOF-after-solidus-in-tagu datau) unexpected-character-after-solidus-in-tagT( u streamu charu Trueu currentTokenu emitCurrentTokenu EOFu tokenQueueu appendu tokenTypesu ungetu dataStateu stateu beforeAttributeNameState( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu selfClosingStartTagState> s u& HTMLTokenizer.selfClosingStartTagStatec C sc | j j d � } | j d d � } | j j i t d d 6| d 6� | j j � | j | _ d S( Nu >u u �u Commentu typeu dataT( u streamu charsUntilu replaceu tokenQueueu appendu tokenTypesu charu dataStateu stateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu bogusCommentStateP s u HTMLTokenizer.bogusCommentStatec C s0 | j j � g } | d d k rv | j | j j � � | d d k r�i t d d 6d d 6| _ | j | _ d Sne| d! d" k rd } x> d) D]6 } | j | j j � � | d* | k r� d+ } Pq� q� W| r�i t d d 6d d 6d d 6d d 6d d 6| _ | j | _ d Sn� | d, d k r�| j d k r�| j j j r�| j j j d- j | j j j k r�d } xP d d d d d d g D]6 } | j | j j � � | d. | k r�d+ } Pq�q�W| r�| j | _ d Sn | j j i t d d 6d d 6� x | r | j j | j � � q W| j | _ d S(/ Ni u -u Commentu typeu u datau du Du ou Ou cu Cu tu Tu yu Yu pu Pu eu Eu Doctypeu nameu publicIdu systemIdu correctu [u Au ParseErroru expected-dashes-or-doctypei����i����Ti����( u du D( u ou O( u cu C( u tu T( u yu Y( u pu P( u eu E( ( u ou O( u cu C( u tu T( u yu Y( u pu P( u eu Ei����Fi����i����i����( u streamu charu appendu tokenTypesu currentTokenu commentStartStateu stateu Trueu Falseu Noneu doctypeStateu parseru treeu openElementsu namespaceu defaultNamespaceu cdataSectionStateu tokenQueueu ungetu popu bogusCommentState( u selfu charStacku matchedu expected( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu markupDeclarationOpenState_ sR % u( HTMLTokenizer.markupDeclarationOpenStatec C s- | j j � } | d k r* | j | _ n� | d k rl | j j i t d d 6d d 6� | j d d 7<n� | d k r� | j j i t d d 6d d 6� | j j | j � | j | _ nm | t k r | j j i t d d 6d d 6� | j j | j � | j | _ n | j d | 7<| j | _ d S( Nu -u u ParseErroru typeu invalid-codepointu datau �u >u incorrect-commentu eof-in-commentT( u streamu charu commentStartDashStateu stateu tokenQueueu appendu tokenTypesu currentTokenu dataStateu EOFu commentStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu commentStartState� s( u HTMLTokenizer.commentStartStatec C s1 | j j � } | d k r* | j | _ n| d k rl | j j i t d d 6d d 6� | j d d 7<n� | d k r� | j j i t d d 6d d 6� | j j | j � | j | _ nq | t k r | j j i t d d 6d d 6� | j j | j � | j | _ n! | j d d | 7<| j | _ d S( Nu -u u ParseErroru typeu invalid-codepointu datau -�u >u incorrect-commentu eof-in-commentT( u streamu charu commentEndStateu stateu tokenQueueu appendu tokenTypesu currentTokenu dataStateu EOFu commentStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu commentStartDashState� s( u# HTMLTokenizer.commentStartDashStatec C s� | j j � } | d k r* | j | _ n� | d k rl | j j i t d d 6d d 6� | j d d 7<nq | t k r� | j j i t d d 6d d 6� | j j | j � | j | _ n! | j d | | j j d � 7<d S( Nu -u u ParseErroru typeu invalid-codepointu datau �u eof-in-comment( u -u T( u streamu charu commentEndDashStateu stateu tokenQueueu appendu tokenTypesu currentTokenu EOFu dataStateu charsUntilu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu commentState� s u HTMLTokenizer.commentStatec C s� | j j � } | d k r* | j | _ n� | d k rx | j j i t d d 6d d 6� | j d d 7<| j | _ nq | t k r� | j j i t d d 6d d 6� | j j | j � | j | _ n! | j d d | 7<| j | _ d S( Nu -u u ParseErroru typeu invalid-codepointu datau -�u eof-in-comment-end-dashT( u streamu charu commentEndStateu stateu tokenQueueu appendu tokenTypesu currentTokenu commentStateu EOFu dataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu commentEndDashState� s u! HTMLTokenizer.commentEndDashStatec C s� | j j � } | d k r= | j j | j � | j | _ n`| d k r� | j j i t d d 6d d 6� | j d d 7<| j | _ n| d k r� | j j i t d d 6d d 6� | j | _ n� | d k r | j j i t d d 6d d 6� | j d | 7<n� | t k rZ| j j i t d d 6d d 6� | j j | j � | j | _ nC | j j i t d d 6d d 6� | j d d | 7<| j | _ d S( Nu >u u ParseErroru typeu invalid-codepointu datau --�u !u, unexpected-bang-after-double-dash-in-commentu -u, unexpected-dash-after-double-dash-in-commentu eof-in-comment-double-dashu unexpected-char-in-commentu --T( u streamu charu tokenQueueu appendu currentTokenu dataStateu stateu tokenTypesu commentStateu commentEndBangStateu EOFu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu commentEndState� s6 u HTMLTokenizer.commentEndStatec C s, | j j � } | d k r= | j j | j � | j | _ n� | d k ri | j d d 7<| j | _ n� | d k r� | j j i t d d 6d d 6� | j d d 7<| j | _ nq | t k r| j j i t d d 6d d 6� | j j | j � | j | _ n! | j d d | 7<| j | _ d S( Nu >u -u datau --!u u ParseErroru typeu invalid-codepointu --!�u eof-in-comment-end-bang-stateT( u streamu charu tokenQueueu appendu currentTokenu dataStateu stateu commentEndDashStateu tokenTypesu commentStateu EOFu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu commentEndBangState� s( u! HTMLTokenizer.commentEndBangStatec C s� | j j � } | t k r* | j | _ n� | t k r� | j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n> | j j i t d d 6d d 6� | j j | � | j | _ d S( Nu ParseErroru typeu! expected-doctype-name-but-got-eofu datau correctu need-space-after-doctypeFT( u streamu charu spaceCharactersu beforeDoctypeNameStateu stateu EOFu tokenQueueu appendu tokenTypesu Falseu currentTokenu dataStateu ungetu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu doctypeState s u HTMLTokenizer.doctypeStatec C s? | j j � } | t k r n | d k r{ | j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n� | d k r� | j j i t d d 6d d 6� d | j d <| j | _ nv | t k r"| j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n | | j d <| j | _ d S( Nu >u ParseErroru typeu+ expected-doctype-name-but-got-right-bracketu datau correctu u invalid-codepointu �u nameu! expected-doctype-name-but-got-eofFT( u streamu charu spaceCharactersu tokenQueueu appendu tokenTypesu Falseu currentTokenu dataStateu stateu doctypeNameStateu EOFu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu beforeDoctypeNameState& s. u$ HTMLTokenizer.beforeDoctypeNameStatec C so | j j � } | t k rG | j d j t � | j d <| j | _ n$| d k r� | j d j t � | j d <| j j | j � | j | _ n� | d k r� | j j i t d d 6d d 6� | j d d 7<| j | _ n� | t k rZ| j j i t d d 6d d 6� d | j d <| j d j t � | j d <| j j | j � | j | _ n | j d | 7<d S( Nu nameu >u u ParseErroru typeu invalid-codepointu datau �u eof-in-doctype-nameu correctFT( u streamu charu spaceCharactersu currentTokenu translateu asciiUpper2Loweru afterDoctypeNameStateu stateu tokenQueueu appendu dataStateu tokenTypesu doctypeNameStateu EOFu Falseu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu doctypeNameState@ s, u HTMLTokenizer.doctypeNameStatec C s� | j j � } | t k r n�| d k rL | j j | j � | j | _ n�| t k r� d | j d <| j j | � | j j i t d d 6d d 6� | j j | j � | j | _ n| d k rd! } x3 d' D]+ } | j j � } | | k r� d } Pq� q� W| r{| j | _ d! Sna | d( k r{d! } x3 d. D]+ } | j j � } | | k r3d } Pq3q3W| r{| j | _ d! Sn | j j | � | j j i t d d 6d d 6i | d 6d 6� d | j d <| j | _ d! S(/ Nu >u correctu ParseErroru typeu eof-in-doctypeu datau pu Pu uu Uu bu Bu lu Lu iu Iu cu Cu su Su yu Yu tu Tu eu Eu mu Mu* expected-space-or-right-bracket-in-doctypeu datavarsF( u pu PT( u uu U( u bu B( u lu L( u iu I( u cu C( ( u uu U( u bu B( u lu L( u iu I( u cu C( u su S( u yu Y( u su S( u tu T( u eu E( u mu M( ( u yu Y( u su S( u tu T( u eu E( u mu M( u streamu charu spaceCharactersu tokenQueueu appendu currentTokenu dataStateu stateu EOFu Falseu ungetu tokenTypesu Trueu afterDoctypePublicKeywordStateu afterDoctypeSystemKeywordStateu bogusDoctypeState( u selfu datau matchedu expected( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu afterDoctypeNameStateY sT u# HTMLTokenizer.afterDoctypeNameStatec C s� | j j � } | t k r* | j | _ n� | d k rw | j j i t d d 6d d 6� | j j | � | j | _ ny | t k r� | j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n | j j | � | j | _ d S( Nu 'u "u ParseErroru typeu unexpected-char-in-doctypeu datau eof-in-doctypeu correct( u 'u "FT( u streamu charu spaceCharactersu" beforeDoctypePublicIdentifierStateu stateu tokenQueueu appendu tokenTypesu ungetu EOFu Falseu currentTokenu dataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu afterDoctypePublicKeywordState� s" u, HTMLTokenizer.afterDoctypePublicKeywordStatec C sg | j j � } | t k r nE| d k rF d | j d <| j | _ n | d k rn d | j d <| j | _ n� | d k r� | j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n� | t k r(| j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n; | j j i t d d 6d d 6� d | j d <| j | _ d S( Nu "u u publicIdu 'u >u ParseErroru typeu unexpected-end-of-doctypeu datau correctu eof-in-doctypeu unexpected-char-in-doctypeFT( u streamu charu spaceCharactersu currentTokenu( doctypePublicIdentifierDoubleQuotedStateu stateu( doctypePublicIdentifierSingleQuotedStateu tokenQueueu appendu tokenTypesu Falseu dataStateu EOFu bogusDoctypeStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu" beforeDoctypePublicIdentifierState� s4 u0 HTMLTokenizer.beforeDoctypePublicIdentifierStatec C s; | j j � } | d k r* | j | _ n | d k rl | j j i t d d 6d d 6� | j d d 7<n� | d k r� | j j i t d d 6d d 6� d | j d <| j j | j � | j | _ nn | t k r&| j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n | j d | 7<d S( Nu "u u ParseErroru typeu invalid-codepointu datau publicIdu �u >u unexpected-end-of-doctypeu correctu eof-in-doctypeFT( u streamu charu! afterDoctypePublicIdentifierStateu stateu tokenQueueu appendu tokenTypesu currentTokenu Falseu dataStateu EOFu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu( doctypePublicIdentifierDoubleQuotedState� s* u6 HTMLTokenizer.doctypePublicIdentifierDoubleQuotedStatec C s; | j j � } | d k r* | j | _ n | d k rl | j j i t d d 6d d 6� | j d d 7<n� | d k r� | j j i t d d 6d d 6� d | j d <| j j | j � | j | _ nn | t k r&| j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n | j d | 7<d S( Nu 'u u ParseErroru typeu invalid-codepointu datau publicIdu �u >u unexpected-end-of-doctypeu correctu eof-in-doctypeFT( u streamu charu! afterDoctypePublicIdentifierStateu stateu tokenQueueu appendu tokenTypesu currentTokenu Falseu dataStateu EOFu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu( doctypePublicIdentifierSingleQuotedState� s* u6 HTMLTokenizer.doctypePublicIdentifierSingleQuotedStatec C s� | j j � } | t k r* | j | _ nZ| d k rX | j j | j � | j | _ n,| d k r� | j j i t d d 6d d 6� d | j d <| j | _ n� | d k r� | j j i t d d 6d d 6� d | j d <| j | _ n� | t k rI| j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n; | j j i t d d 6d d 6� d | j d <| j | _ d S( Nu >u "u ParseErroru typeu unexpected-char-in-doctypeu datau u systemIdu 'u eof-in-doctypeu correctFT( u streamu charu spaceCharactersu- betweenDoctypePublicAndSystemIdentifiersStateu stateu tokenQueueu appendu currentTokenu dataStateu tokenTypesu( doctypeSystemIdentifierDoubleQuotedStateu( doctypeSystemIdentifierSingleQuotedStateu EOFu Falseu bogusDoctypeStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu! afterDoctypePublicIdentifierState� s6 u/ HTMLTokenizer.afterDoctypePublicIdentifierStatec C s8 | j j � } | t k r n| d k rL | j j | j � | j | _ n� | d k rt d | j d <| j | _ n� | d k r� d | j d <| j | _ n� | t k r� | j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n; | j j i t d d 6d d 6� d | j d <| j | _ d S( Nu >u "u u systemIdu 'u ParseErroru typeu eof-in-doctypeu datau correctu unexpected-char-in-doctypeFT( u streamu charu spaceCharactersu tokenQueueu appendu currentTokenu dataStateu stateu( doctypeSystemIdentifierDoubleQuotedStateu( doctypeSystemIdentifierSingleQuotedStateu EOFu tokenTypesu Falseu bogusDoctypeStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu- betweenDoctypePublicAndSystemIdentifiersState s. u; HTMLTokenizer.betweenDoctypePublicAndSystemIdentifiersStatec C s� | j j � } | t k r* | j | _ n� | d k rw | j j i t d d 6d d 6� | j j | � | j | _ ny | t k r� | j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n | j j | � | j | _ d S( Nu 'u "u ParseErroru typeu unexpected-char-in-doctypeu datau eof-in-doctypeu correct( u 'u "FT( u streamu charu spaceCharactersu" beforeDoctypeSystemIdentifierStateu stateu tokenQueueu appendu tokenTypesu ungetu EOFu Falseu currentTokenu dataStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu afterDoctypeSystemKeywordState% s" u, HTMLTokenizer.afterDoctypeSystemKeywordStatec C sg | j j � } | t k r nE| d k rF d | j d <| j | _ n | d k rn d | j d <| j | _ n� | d k r� | j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n� | t k r(| j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n; | j j i t d d 6d d 6� d | j d <| j | _ d S( Nu "u u systemIdu 'u >u ParseErroru typeu unexpected-char-in-doctypeu datau correctu eof-in-doctypeFT( u streamu charu spaceCharactersu currentTokenu( doctypeSystemIdentifierDoubleQuotedStateu stateu( doctypeSystemIdentifierSingleQuotedStateu tokenQueueu appendu tokenTypesu Falseu dataStateu EOFu bogusDoctypeStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu" beforeDoctypeSystemIdentifierState9 s4 u0 HTMLTokenizer.beforeDoctypeSystemIdentifierStatec C s; | j j � } | d k r* | j | _ n | d k rl | j j i t d d 6d d 6� | j d d 7<n� | d k r� | j j i t d d 6d d 6� d | j d <| j j | j � | j | _ nn | t k r&| j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n | j d | 7<d S( Nu "u u ParseErroru typeu invalid-codepointu datau systemIdu �u >u unexpected-end-of-doctypeu correctu eof-in-doctypeFT( u streamu charu! afterDoctypeSystemIdentifierStateu stateu tokenQueueu appendu tokenTypesu currentTokenu Falseu dataStateu EOFu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu( doctypeSystemIdentifierDoubleQuotedStateV s* u6 HTMLTokenizer.doctypeSystemIdentifierDoubleQuotedStatec C s; | j j � } | d k r* | j | _ n | d k rl | j j i t d d 6d d 6� | j d d 7<n� | d k r� | j j i t d d 6d d 6� d | j d <| j j | j � | j | _ nn | t k r&| j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n | j d | 7<d S( Nu 'u u ParseErroru typeu invalid-codepointu datau systemIdu �u >u unexpected-end-of-doctypeu correctu eof-in-doctypeFT( u streamu charu! afterDoctypeSystemIdentifierStateu stateu tokenQueueu appendu tokenTypesu currentTokenu Falseu dataStateu EOFu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu( doctypeSystemIdentifierSingleQuotedStaten s* u6 HTMLTokenizer.doctypeSystemIdentifierSingleQuotedStatec C s� | j j � } | t k r n� | d k rL | j j | j � | j | _ n� | t k r� | j j i t d d 6d d 6� d | j d <| j j | j � | j | _ n. | j j i t d d 6d d 6� | j | _ d S( Nu >u ParseErroru typeu eof-in-doctypeu datau correctu unexpected-char-in-doctypeFT( u streamu charu spaceCharactersu tokenQueueu appendu currentTokenu dataStateu stateu EOFu tokenTypesu Falseu bogusDoctypeStateu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu! afterDoctypeSystemIdentifierState� s u/ HTMLTokenizer.afterDoctypeSystemIdentifierStatec C s | j j � } | d k r= | j j | j � | j | _ n> | t k r{ | j j | � | j j | j � | j | _ n d S( Nu >T( u streamu charu tokenQueueu appendu currentTokenu dataStateu stateu EOFu ungetu True( u selfu data( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu bogusDoctypeState� s u HTMLTokenizer.bogusDoctypeStatec C se g } x� | j | j j d � � | j | j j d � � | j j � } | t k rZ Pq | d k sl t � | d d d � d k r� | d d d � | d <Pq | j | � q d j | � } | j d � } | d k r*x6 t | � D]( } | j j i t d d 6d d 6� q� W| j d d � } n | rU| j j i t d d 6| d 6� n | j | _ d S( Nu ]u >i i u ]]u u i u ParseErroru typeu invalid-codepointu datau �u Charactersi����i����i����i����i����T( u appendu streamu charsUntilu charu EOFu AssertionErroru joinu countu rangeu tokenQueueu tokenTypesu replaceu dataStateu stateu True( u selfu datau charu nullCountu i( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu cdataSectionState� s0 u HTMLTokenizer.cdataSectionStateNTF(P u __name__u __module__u __qualname__u __doc__u Noneu Trueu __init__u __iter__u consumeNumberEntityu Falseu consumeEntityu processEntityInAttributeu emitCurrentTokenu dataStateu entityDataStateu rcdataStateu characterReferenceInRcdatau rawtextStateu scriptDataStateu plaintextStateu tagOpenStateu closeTagOpenStateu tagNameStateu rcdataLessThanSignStateu rcdataEndTagOpenStateu rcdataEndTagNameStateu rawtextLessThanSignStateu rawtextEndTagOpenStateu rawtextEndTagNameStateu scriptDataLessThanSignStateu scriptDataEndTagOpenStateu scriptDataEndTagNameStateu scriptDataEscapeStartStateu scriptDataEscapeStartDashStateu scriptDataEscapedStateu scriptDataEscapedDashStateu scriptDataEscapedDashDashStateu" scriptDataEscapedLessThanSignStateu scriptDataEscapedEndTagOpenStateu scriptDataEscapedEndTagNameStateu scriptDataDoubleEscapeStartStateu scriptDataDoubleEscapedStateu scriptDataDoubleEscapedDashStateu$ scriptDataDoubleEscapedDashDashStateu( scriptDataDoubleEscapedLessThanSignStateu scriptDataDoubleEscapeEndStateu beforeAttributeNameStateu attributeNameStateu afterAttributeNameStateu beforeAttributeValueStateu attributeValueDoubleQuotedStateu attributeValueSingleQuotedStateu attributeValueUnQuotedStateu afterAttributeValueStateu selfClosingStartTagStateu bogusCommentStateu markupDeclarationOpenStateu commentStartStateu commentStartDashStateu commentStateu commentEndDashStateu commentEndStateu commentEndBangStateu doctypeStateu beforeDoctypeNameStateu doctypeNameStateu afterDoctypeNameStateu afterDoctypePublicKeywordStateu" beforeDoctypePublicIdentifierStateu( doctypePublicIdentifierDoubleQuotedStateu( doctypePublicIdentifierSingleQuotedStateu! afterDoctypePublicIdentifierStateu- betweenDoctypePublicAndSystemIdentifiersStateu afterDoctypeSystemKeywordStateu" beforeDoctypeSystemIdentifierStateu( doctypeSystemIdentifierDoubleQuotedStateu( doctypeSystemIdentifierSingleQuotedStateu! afterDoctypeSystemIdentifierStateu bogusDoctypeStateu cdataSectionState( u __locals__( ( u __class__u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu HTMLTokenizer s� HP # 7 " - 3 u HTMLTokenizerN( u __future__u absolute_importu divisionu unicode_literalsu unichru chru NameErroru collectionsu dequeu constantsu spaceCharactersu entitiesu asciiLettersu asciiUpper2Loweru digitsu hexDigitsu EOFu tokenTypesu tagTokenTypesu replacementCharactersu inputstreamu HTMLInputStreamu trieu Trieu entitiesTrieu objectu HTMLTokenizer( ( ( u7 /tmp/pip-zej_zi-build/pip/_vendor/html5lib/tokenizer.pyu <module> s