PCREPARTIAL(3) - phpMan

Command: man perldoc info search(apropos)  


PCREPARTIAL(3)                       Library Functions Manual                      PCREPARTIAL(3)



NAME
       PCRE - Perl-compatible regular expressions

PARTIAL MATCHING IN PCRE

       In normal use of PCRE, if the subject string that is passed to a matching function matches
       as far as it goes, but is too short to match the  entire  pattern,  PCRE_ERROR_NOMATCH  is
       returned.  There are circumstances where it might be helpful to distinguish this case from
       other cases in which there is no match.

       Consider, for example, an application where a human is required to  type  in  data  for  a
       field  with specific formatting requirements. An example might be a date in the form ddmm-
       myy, defined by this pattern:

         ^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$

       If the application sees the user's keystrokes one by one, and can check that what has been
       typed  so  far  is potentially valid, it is able to raise an error as soon as a mistake is
       made, by beeping and not reflecting the character that has been typed, for  example.  This
       immediate  feedback  is  likely to be a better user interface than a check that is delayed
       until the entire string has been entered. Partial matching can also  be  useful  when  the
       subject string is very long and is not all available at once.

       PCRE  supports  partial  matching  by means of the PCRE_PARTIAL_SOFT and PCRE_PARTIAL_HARD
       options, which can be set when calling any of the matching functions. For  backwards  com-
       patibility,  PCRE_PARTIAL  is  a  synonym  for PCRE_PARTIAL_SOFT. The essential difference
       between the two options is whether or not a partial match is preferred to  an  alternative
       complete  match,  though the details differ between the two types of matching function. If
       both options are set, PCRE_PARTIAL_HARD takes precedence.

       If you want to use partial matching  with  just-in-time  optimized  code,  you  must  call
       pcre_study(), pcre16_study() or  pcre32_study() with one or both of these options:

         PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE
         PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE

       PCRE_STUDY_JIT_COMPILE  should  also be set if you are going to run non-partial matches on
       the same pattern. If the appropriate JIT study mode has not been  set  for  a  match,  the
       interpretive matching code is used.

       Setting  a  partial  matching  option  disables two of PCRE's standard optimizations. PCRE
       remembers the last literal data unit in a pattern, and abandons matching immediately if it
       is  not  present  in  the  subject  string. This optimization cannot be used for a subject
       string that might match only partially. If the pattern was studied, PCRE knows the minimum
       length  of  a matching string, and does not bother to run the matching function on shorter
       strings. This optimization is also disabled for partial matching.

PARTIAL MATCHING USING pcre_exec() OR pcre[16|32]_exec()

       A partial match occurs during a call to pcre_exec() or pcre[16|32]_exec() when the end  of
       the  subject  string  is  reached  successfully, but matching cannot continue because more
       characters are needed. However, at least one character  in  the  subject  must  have  been
       inspected.  This  character  need  not  form  part of the final matched string; lookbehind
       assertions and the \K escape sequence provide ways of  inspecting  characters  before  the
       start of a matched substring. The requirement for inspecting at least one character exists
       because an empty string can always be matched; without  such  a  restriction  there  would
       always be a partial match of an empty string at the end of the subject.

       If  there  are  at least two slots in the offsets vector when a partial match is returned,
       the first slot is set to the offset of the earliest character that was inspected. For con-
       venience,  the second offset points to the end of the subject so that a substring can eas-
       ily be identified.

       For the majority of patterns, the first offset  identifies  the  start  of  the  partially
       matched  string. However, for patterns that contain lookbehind assertions, or \K, or begin
       with \b or \B, earlier characters have been inspected while carrying out  the  match.  For
       example:

         /(?<=abc)123/

       This  pattern matches "123", but only if it is preceded by "abc". If the subject string is
       "xyzabc12", the offsets after a partial match are for the substring "abc12",  because  all
       these  characters  are needed if another match is tried with extra characters added to the
       subject.

       What happens when a partial match is identified depends on which of the two partial match-
       ing options are set.

   PCRE_PARTIAL_SOFT WITH pcre_exec() OR pcre[16|32]_exec()

       If  PCRE_PARTIAL_SOFT  is  set when pcre_exec() or pcre[16|32]_exec() identifies a partial
       match, the partial match is remembered, but matching continues as normal, and other alter-
       natives in the pattern are tried. If no complete match can be found, PCRE_ERROR_PARTIAL is
       returned instead of PCRE_ERROR_NOMATCH.

       This option is "soft" because it prefers a complete match over a partial match.   All  the
       various  matching  items  in a pattern behave as if the subject string is potentially com-
       plete. For example, \z, \Z, and $ match at the end of the subject, as normal, and  for  \b
       and \B the end of the subject is treated as a non-alphanumeric.

       If  there  is  more than one partial match, the first one that was found provides the data
       that is returned. Consider this pattern:

         /123\w+X|dogY/

       If this is matched against the subject  string  "abc123dog",  both  alternatives  fail  to
       match,  but  the  end  of the subject is reached during matching, so PCRE_ERROR_PARTIAL is
       returned. The offsets are set to 3 and 9, identifying "123dog" as the first partial  match
       that  was found. (In this example, there are two partial matches, because "dog" on its own
       partially matches the second alternative.)

   PCRE_PARTIAL_HARD WITH pcre_exec() OR pcre[16|32]_exec()

       If PCRE_PARTIAL_HARD is set for pcre_exec() or pcre[16|32]_exec(),  PCRE_ERROR_PARTIAL  is
       returned  as  soon  as a partial match is found, without continuing to search for possible
       complete matches. This option is "hard" because it prefers an earlier partial match over a
       later complete match. For this reason, the assumption is made that the end of the supplied
       subject string may not be the true end of the available data, and so, if \z, \Z,  \b,  \B,
       or $ are encountered at the end of the subject, the result is PCRE_ERROR_PARTIAL, provided
       that at least one character in the subject has been inspected.

       Setting PCRE_PARTIAL_HARD also affects the  way  UTF-8  and  UTF-16  subject  strings  are
       checked for validity. Normally, an invalid sequence causes the error PCRE_ERROR_BADUTF8 or
       PCRE_ERROR_BADUTF16. However, in the special case of a truncated character at the  end  of
       the  subject,  PCRE_ERROR_SHORTUTF8  or  PCRE_ERROR_SHORTUTF16  is returned when PCRE_PAR-
       TIAL_HARD is set.

   Comparing hard and soft partial matching

       The difference between the two partial matching options can be illustrated  by  a  pattern
       such as:

         /dog(sbody)?/

       This  matches  either "dog" or "dogsbody", greedily (that is, it prefers the longer string
       if possible). If it is matched against the string "dog" with PCRE_PARTIAL_SOFT, it  yields
       a  complete  match  for  "dog".  However,  if  PCRE_PARTIAL_HARD  is  set,  the  result is
       PCRE_ERROR_PARTIAL. On the other hand, if the pattern is made ungreedy the result is  dif-
       ferent:

         /dog(sbody)??/

       In this case the result is always a complete match because that is found first, and match-
       ing never continues after finding a complete match. It might  be  easier  to  follow  this
       explanation by thinking of the two patterns like this:

         /dog(sbody)?/    is the same as  /dogsbody|dog/
         /dog(sbody)??/   is the same as  /dog|dogsbody/

       The  second  pattern  will never match "dogsbody", because it will always find the shorter
       match first.

PARTIAL MATCHING USING pcre_dfa_exec() OR pcre[16|32]_dfa_exec()

       The DFA functions move along the subject string character by character, without backtrack-
       ing,  searching  for  all  possible  matches  simultaneously. If the end of the subject is
       reached before the end of the pattern, there is the possibility of a partial match,  again
       provided that at least one character has been inspected.

       When  PCRE_PARTIAL_SOFT  is set, PCRE_ERROR_PARTIAL is returned only if there have been no
       complete matches. Otherwise, the complete matches are  returned.   However,  if  PCRE_PAR-
       TIAL_HARD  is set, a partial match takes precedence over any complete matches. The portion
       of the string that was inspected when the longest partial match was found is  set  as  the
       first matching string, provided there are at least two slots in the offsets vector.

       Because  the DFA functions always search for all possible matches, and there is no differ-
       ence between greedy and ungreedy repetition, their behaviour is different from  the  stan-
       dard  functions  when  PCRE_PARTIAL_HARD is set. Consider the string "dog" matched against
       the ungreedy pattern shown above:

         /dog(sbody)??/

       Whereas the standard functions stop as soon as they find the complete match for "dog", the
       DFA  functions  also  find  the  partial  match  for  "dogsbody",  and so return that when
       PCRE_PARTIAL_HARD is set.

PARTIAL MATCHING AND WORD BOUNDARIES

       If a pattern ends with one of sequences \b or \B, which test for word boundaries,  partial
       matching with PCRE_PARTIAL_SOFT can give counter-intuitive results. Consider this pattern:

         /\bcat\b/

       This matches "cat", provided there is a word boundary at either end. If the subject string
       is "the cat", the comparison of the final "t"  with  a  following  character  cannot  take
       place, so a partial match is found. However, normal matching carries on, and \b matches at
       the end of the subject when the last character is a letter, so a complete match is  found.
       The  result,  therefore,  is  not PCRE_ERROR_PARTIAL. Using PCRE_PARTIAL_HARD in this case
       does yield PCRE_ERROR_PARTIAL, because then the partial match takes precedence.

FORMERLY RESTRICTED PATTERNS

       For releases of PCRE prior to 8.00, because of the way certain internal optimizations were
       implemented in the pcre_exec() function, the PCRE_PARTIAL option (predecessor of PCRE_PAR-
       TIAL_SOFT) could not be used with all patterns. From release 8.00  onwards,  the  restric-
       tions no longer apply, and partial matching with can be requested for any pattern.

       Items  that  were formerly restricted were repeated single characters and repeated metase-
       quences. If PCRE_PARTIAL was set for a pattern that did not conform to  the  restrictions,
       pcre_exec()  returned  the  error  code PCRE_ERROR_BADPARTIAL (-13). This error code is no
       longer in use. The PCRE_INFO_OKPARTIAL call to pcre_fullinfo() to find out if  a  compiled
       pattern can be used for partial matching now always returns 1.

EXAMPLE OF PARTIAL MATCHING USING PCRETEST

       If the escape sequence \P is present in a pcretest data line, the PCRE_PARTIAL_SOFT option
       is used for the match. Here is a run of pcretest that uses the date example quoted above:

           re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/
         data> 25jun04\P
          0: 25jun04
          1: jun
         data> 25dec3\P
         Partial match: 23dec3
         data> 3ju\P
         Partial match: 3ju
         data> 3juj\P
         No match
         data> j\P
         No match

       The first data string is matched completely, so pcretest shows the matched substrings. The
       remaining  four  strings  do not match the complete pattern, but the first two are partial
       matches. Similar output is obtained if DFA matching is used.

       If the escape sequence \P is present more than once in a pcretest data line, the PCRE_PAR-
       TIAL_HARD option is set for the match.

MULTI-SEGMENT MATCHING WITH pcre_dfa_exec() OR pcre[16|32]_dfa_exec()

       When  a partial match has been found using a DFA matching function, it is possible to con-
       tinue the match by providing additional subject data and calling the function  again  with
       the  same  compiled regular expression, this time setting the PCRE_DFA_RESTART option. You
       must pass the same working space as before, because this is where details of the  previous
       partial  match are stored. Here is an example using pcretest, using the \R escape sequence
       to set the PCRE_DFA_RESTART option (\D specifies the use of the DFA matching function):

           re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/
         data> 23ja\P\D
         Partial match: 23ja
         data> n05\R\D
          0: n05

       The first call has "23ja" as the subject, and requests partial matching; the  second  call
       has  "n05" as the subject for the continued (restarted) match.  Notice that when the match
       is complete, only the last part is shown; PCRE does not retain the  previously  partially-
       matched string. It is up to the calling program to do that if it needs to.

       You  can  set  the PCRE_PARTIAL_SOFT or PCRE_PARTIAL_HARD options with PCRE_DFA_RESTART to
       continue partial matching over multiple segments. This facility can be used to  pass  very
       long subject strings to the DFA matching functions.

MULTI-SEGMENT MATCHING WITH pcre_exec() OR pcre[16|32]_exec()

       From  release  8.00,  the standard matching functions can also be used to do multi-segment
       matching. Unlike the DFA functions, it is not possible to restart the previous match  with
       a new segment of data. Instead, new data must be added to the previous subject string, and
       the entire match re-run, starting from the point where the partial match occurred. Earlier
       data can be discarded.

       It  is  best to use PCRE_PARTIAL_HARD in this situation, because it does not treat the end
       of a segment as the end of the subject when matching \z, \Z, \b, \B, and  $.  Consider  an
       unanchored pattern that matches dates:

           re> /\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d/
         data> The date is 23ja\P\P
         Partial match: 23ja

       At  this  stage,  an application could discard the text preceding "23ja", add on text from
       the next segment, and call the matching function again. Unlike the DFA matching functions,
       the  entire  matching  string  must always be available, and the complete matching process
       occurs for each call, so more memory and more processing time is needed.

       Note: If the pattern contains lookbehind assertions, or \K, or starts with \b or  \B,  the
       string that is returned for a partial match includes characters that precede the partially
       matched string itself, because these must be retained when adding on more characters for a
       subsequent  matching  attempt.  However, in some cases you may need to retain even earlier
       characters, as discussed in the next section.

ISSUES WITH MULTI-SEGMENT MATCHING

       Certain types of pattern may give problems with multi-segment matching, whichever matching
       function is used.

       1.  If  the  pattern  contains  a  test  for the beginning of a line, you need to pass the
       PCRE_NOTBOL option when the subject string for any call does start at the beginning  of  a
       line.  There is also a PCRE_NOTEOL option, but in practice when doing multi-segment match-
       ing you should be using PCRE_PARTIAL_HARD, which includes the effect of PCRE_NOTEOL.

       2. Lookbehind assertions that have already been obeyed are catered for in the offsets that
       are  returned  for  a  partial  match. However a lookbehind assertion later in the pattern
       could require even earlier characters to be inspected. You can handle this case  by  using
       the  PCRE_INFO_MAXLOOKBEHIND option of the pcre_fullinfo() or pcre[16|32]_fullinfo() func-
       tions to obtain the length of the largest lookbehind in the pattern. This length is  given
       in  characters,  not  bytes. If you always retain at least that many characters before the
       partially matched string, all should be well. (Of course, near the start of  the  subject,
       fewer characters may be present; in that case all characters should be retained.)

       3.  Because a partial match must always contain at least one character, what might be con-
       sidered a partial match of an empty string actually gives a "no match" result.  For  exam-
       ple:

           re> /c(?<=abc)x/
         data> ab\P
         No match

       If  the  next  segment  begins "cx", a match should be found, but this will only happen if
       characters from the previous segment are retained. For this reason, a  "no  match"  result
       should  be  interpreted  as  "partial  match of an empty string" when the pattern contains
       lookbehinds.

       4. Matching a subject string that is split into multiple segments may not  always  produce
       exactly the same result as matching over one single long string, especially when PCRE_PAR-
       TIAL_SOFT is used. The section "Partial Matching and Word Boundaries" above  describes  an
       issue  that arises if the pattern ends with \b or \B. Another kind of difference may occur
       when there are multiple matching possibilities, because (for PCRE_PARTIAL_SOFT) a  partial
       match result is given only when there are no completed matches. This means that as soon as
       the shortest match has been found, continuation to a new subject segment is no longer pos-
       sible. Consider again this pcretest example:

           re> /dog(sbody)?/
         data> dogsb\P
          0: dog
         data> do\P\D
         Partial match: do
         data> gsb\R\P\D
          0: g
         data> dogsbody\D
          0: dogsbody
          1: dog

       The first data line passes the string "dogsb" to a standard matching function, setting the
       PCRE_PARTIAL_SOFT option. Although the string is  a  partial  match  for  "dogsbody",  the
       result  is  not  PCRE_ERROR_PARTIAL, because the shorter string "dog" is a complete match.
       Similarly, when the subject is presented to a DFA matching function in several parts ("do"
       and  "gsb"  being  the first two) the match stops when "dog" has been found, and it is not
       possible to continue.  On the other hand, if "dogsbody" is presented as a single string, a
       DFA matching function finds both matches.

       Because of these problems, it is best to use PCRE_PARTIAL_HARD when matching multi-segment
       data. The example above then behaves differently:

           re> /dog(sbody)?/
         data> dogsb\P\P
         Partial match: dogsb
         data> do\P\D
         Partial match: do
         data> gsb\R\P\P\D
         Partial match: gsb

       5. Patterns that contain alternatives at the top level which do not  all  start  with  the
       same  pattern  item  may  not work as expected when PCRE_DFA_RESTART is used. For example,
       consider this pattern:

         1234|3789

       If the first part of the subject is "ABC123", a partial match of the first alternative  is
       found  at  offset  3. There is no partial match for the second alternative, because such a
       match does not start at the same point in the subject string. Attempting to continue  with
       the string "7890" does not yield a match because only those alternatives that match at one
       point in the subject are remembered. The problem arises because the start  of  the  second
       alternative  matches  within the first alternative. There is no problem with anchored pat-
       terns or patterns such as:

         1234|ABCD

       where no string can be a partial match for both alternatives. This is not a problem  if  a
       standard matching function is used, because the entire match has to be rerun each time:

           re> /1234|3789/
         data> ABC123\P\P
         Partial match: 123
         data> 1237890
          0: 3789

       Of  course, instead of using PCRE_DFA_RESTART, the same technique of re-running the entire
       match can also be used with the DFA matching functions. Another  possibility  is  to  work
       with  two  buffers.  If a partial match at offset n in the first buffer is followed by "no
       match" when PCRE_DFA_RESTART is used on the second buffer, you can then try  a  new  match
       starting at offset n+1 in the first buffer.

AUTHOR

       Philip Hazel
       University Computing Service
       Cambridge CB2 3QH, England.

REVISION

       Last updated: 24 June 2012
       Copyright (c) 1997-2012 University of Cambridge.



PCRE 8.31                                  24 June 2012                            PCREPARTIAL(3)

Generated by $Id: phpMan.php,v 4.55 2007/09/05 04:42:51 chedong Exp $ Author: Che Dong
On Apache/2.4.6 (CentOS)
Under GNU General Public License
2024-12-24 01:15 @127.0.0.1 CrawledBy Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
Valid XHTML 1.0!Valid CSS!