Enum Token

pub enum Token<'t> {
    Text(&'t [u8]),
    Control(&'t [u8]),
    Sequence(Control, &'t [u8]),
}

A text or control sequence token.

Variants

Text(&'t [u8])

One or more UTF-8 characters excluding C0 and C1 controls.

Control(&'t [u8])

A C0 or C1 control that doesn’t start or end a sequence. This token always has one byte of character data.

Sequence(Control, &'t [u8])

A control sequence with its initial control and subsequent payload.

Implementations

impl Token<'_>

pub fn control(&self) -> Option<Control>

Get this token’s control.

pub fn data(&self) -> &[u8]

Get this token’s character data.

The length of the returned byte slice varies for text and sequence tokens. It always is 1, however, for the control token.

Trait Implementations

impl<'t> Clone for Token<'t>

fn clone(&self) -> Token<'t>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token<'_>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'t> PartialEq for Token<'t>

fn eq(&self, other: &Token<'t>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<'t> Eq for Token<'t>

impl<'t> StructuralPartialEq for Token<'t>