Markup Highlighting

Syntax Highlighting – highlights syntactic text constructs using various colors, fonts, and styles. It is usually used to make it easier to read the source text of computer programs, to improve visual perception.

Wikipedia

At the request of my readers, which fortunately coincided with my intentions, I decided to remove the example.detail field, which stored text in RTF format. Instead, add a text field example.snippet, which will store the code snippet with comments. But since I already managed to put examples into the database, I had to make a button and a small script to transfer the text.

RTF -> TXT converter

procedure frmExampleView_Button1_OnClick(Sender: TObject; var Cancel: boolean);
var
  tmpDataSet:TDataSet;
begin
  SQLQuery('select * from example',tmpDataSet);
  while not tmpDataSet.EOF do
  begin
    frmExampleView.redExample.TextRTF := tmpDataSet.FieldByName('detail').asString;
    SQLExecute('UPDATE example SET snippet = '''+ escape_special_characters( frmExampleView.redExample.Text )+''' WHERE id = '+tmpDataSet.FieldByName('id').asString);
    tmpDataSet.Next;
  end;
  tmpDataSet.Free;
  frmExampleView.redExample.Clear;
  ShowMessage('Transfer complete!');
end;
Code language: Delphi (delphi)

The code uses the fact that the example view form already has a RichEdit component, which is used for conversion. If you need to create a conversion function, you will have to create a separate instance of the TdbRichEdit class.

The frmExampleView.Button1 button is temporary, I will remove it in the next version of the program.

Edit form

To edit the example, we will use the usual TdbMemo multi-line editing field, in which we will activate the display of a vertical scroll bar (ScrollBars = ssVertical property).

View form

The view form remains the same, using the RichEdit component, which has toolbars and rulers disabled (Properties Toolbar1.Show = False, Toolbar2.Show = False, Toolbar3.Show = False, Ruler = False).

I decided to implement code highlighting through the HTML format. It seemed to me that this is a simpler, more transparent and universal solution than being tied to the capabilities of TdbRichEdit or the RTF format.

Unfortunately, in MVDB, RichEdit does not implement a method for adding text in HTML format, this can only be done through loading a file and the LoadHTML() method, so another small script has been added to the collection of extensions:

procedure RichEdit_SetHTMLText(AEdit:TdbRichEdit; AText:string);
// load HTML into editor
var
   tmpFileName:string;
   tmpMemo: TdbMemo;
begin
   tmpFileName := GetTempFileName; // get temporary file name
   tmpMemo := TdbMemo.Create(nil); // create a component for writing a text file in UTF-8 format
   tmpMemo.Text := AText;
   tmpMemo.SaveToFileUTF8(tmpFileName ); // save file
   AEdit.Clear;
   AEdit.LoadHTML(tmpFileName); // upload file as HTML
   DeleteFile(tmpFileName); // delete temporary file
end;
Code language: Delphi (delphi)

The UserApp_LoadExample() data loading method has also changed: it uses the RichEdit_SetHTMLText() described above, as well as the function of converting text to HTML format with CodeHL_TextToHTML() syntax highlighting.

procedure UserApp_LoadExample;
// loading an example from the database
var
  tmpData:string;
  tmpTableName:string;
  tmpTableID:string;
  tmpText:string;
begin
  frmExampleView.edtCaption.Text := '';
  frmExampleView.redExample.Clear;
  if ActiveGrid <> nil then
  begin
  tmpTableName := Grid_GetTableName( ActiveGrid );
  if tmpTableName <> '' then
  begin
    tmpTableID := IntToStr(ActiveGrid.dbItemID);
    try
      // get existing ID
      IDExample := SQLExecute('SELECT id_example FROM '+tmpTableName+' WHERE id = '+tmpTableID );
    except // and if the ID does not exist, it will be -1
      IDExample := -1;
    end;
    // load formatted text
    tmpText := SQLExecute('SELECT snippet FROM example WHERE id = '+IntToStr(IDExample));
    RichEdit_SetHTMLText( frmExampleView.redExample, CodeHL_TextToHTML( tmpText ));
    // load title
    frmExampleView.edtCaption.Text := SQLExecute('SELECT caption FROM example WHERE id = '+IntToStr(IDExample));
  end;
  end;
end;
Code language: Delphi (delphi)

TXT -> HTML converter

In fact, the converter is a parser that looks through the source text character by character. When the end of a word (one of the delimiter characters) is detected, a check is made for the beginning of the linear mode, and then the word is searched in arrays of keywords and special text formatting actions are performed – adding HTML sequences.

Line Mode

Linear mode is used when detecting comments as well as string literals. Linear mode disables parsing of text into words until a special character (or multiple characters) is found to end linear mode.

Three kinds of comments are allowed in ObjectPascal and one way of defining string literals.

Keywords

There are two groups of keywords in ObjectPascal: language keywords and standard classes. Each group has its own color and selection style. In addition to keywords, numeric values are usually highlighted. I implemented all the described functionality in the CodeHL_CheckWord() procedure.

procedure CodeHL_CheckWord(var AWord:string );
// checking the word, adding tags
var
  s:string;
  i: integer;
  j: integer
begin
  s := LowerCase(Trim(AWord));
  if (s = '') or (s = ',') then
    exit;
  // if the word is a number
  if ValidInt(s) or ValidFloat(s) then
  begin
    AWord := '<font color="#FF00FF"><b>'+AWord+'</b></font>';
    exit;
  end;
  // if word is in list 1
  for i:=0 to Length(CHL_KeyWord1) - 1 do
  begin
    j := CompareText(s,CHL_KeyWord1[i]);
    if j = 0 then
    begin
      AWord := '<font color="#005F00"><b>'+AWord+'</b></font>';
      exit;
    end
    else
    if j = -1 then
    begin
      break
    end;
  end;
  // if word is in list 2
  for i:=0 to Length(CHL_KeyWord2) - 1 do
  begin
    j := CompareText(s,CHL_KeyWord2[i]);
    if j = 0 then
    begin
      AWord := '<font color="#000000"><b>'+AWord+'</b></font>';
      exit;
    end
    else
    if j = -1 then
    begin
      break
    end;
  end;
end;
Code language: Delphi (delphi)

The main code highlighting function CodeHL_TextToHTML(), in addition to the actions described above, replaces the characters used for markup with special character sequences. And all text is wrapped in tags that preserve spaces and line breaks when displaying data.

function CodeHL_TextToHTML(AText: string):string;
// adding highlighted HTML tags to text
var
  i: integer;
  j: integer
  tmpWord:string;
  tmpchar:string;
  s1, s2: string;
  tmpIndex: integer;
begin
  tmpIndex := -1;
  Result := '';
  tmpWord := '';
  for i := 1 to Length(AText) do
  begin
    tmpChar := AText[i];
    // replacement to insert into HTML
    case tmpChar of
    '<': tmpChar:='&lt;';
    '>': tmpChar:='&gt;';
    '&': tmpChar:='&amp;';
    chr(10): tmpChar:='';
    end;
    // linear mode - this disables character-by-character analysis until a trigger sequence is detected
    if tmpIndex <> -1 then
    begin
      s1 := CHL_LM_End[tmpIndex];
      // terminating sequence found?
      if copy(AText,i,Length(s1) ) = s1 then
      begin
        tmpWord := tmpWord + S1;
        // if the terminating sequence is more than one character, then we make an adjustment
        if Length(S1) > 1 then
          delete( AText, i+1, Length(s1) - 1 );
        //
        if CHL_LM_Italic[tmpIndex] = '1' then
        begin
          s1 := '<i>';
          s2 := '</i>';
        end
        else
        begin
          s1 := '';
          s2 := '';
        end;
        Result := Result + '<font color="'+CHL_LM_Color[tmpIndex]+'">'+S1+ tmpWord +S2+ '</font>';
        tmpIndex := -1;
        tmpWord := '';
      end
      else
        tmpWord := tmpWord + tmpChar;
      continue;
    end;
    // end of word
    if AText[i] in [' ','{','}','/','\',')','(',';',':','"','''' ,'.',chr(13)] then
    begin
      // check for start of linear mode
      for j:=0 to Length( CHL_LM_Begin )-1 do
      begin
        if copy( AText, i, Length(CHL_LM_Begin[j]) ) = CHL_LM_Begin[j] then
        begin
          CodeHL_CheckWord(tmpWord);
          Result := Result + tmpWord;
          tmpIndex := j;
          continue;
        end;
      end;
      // check for words
      CodeHL_CheckWord(tmpWord);
      Result := Result + tmpWord;
      Result := Result + AText[i]; // delimiter character that was at the end of the word
      tmpWord := '';
    end
    else
      tmpWord := tmpWord + tmpChar;
  end;
  Result := Result + tmpWord;
  Result := '<pre>'+Result+'</pre>';
end;
Code language: Delphi (delphi)

While the function looks rough in places, overall it does its job and has acceptable performance: the delay in displaying data is about half a second, including the creation of a temporary file for loading HTML.

Data

All the data necessary for work is stored in arrays, which are filled by the CodeHL_Init() procedure. This can later be used to extend functionality by adding different languages ​​for highlighting. Within the framework of the program described in this article, you will definitely need the SQL language, since some of the tasks relate specifically to the use of SQL queries and commands, and it would be nice to highlight the syntax of this language.

var
  CHL_KeyWord1 : array of string; // language keywords
  CHL_KeyWord2 : array of string; // standard types

  // line mode parameters
  CHL_LM_Begin: array of string; // start character
  CHL_LM_End: array of string; // end character
  CHL_LM_Color: array of string; // color as code
  CHL_LM_Italic: array of string; // sign of the inclination of the text

procedure CodeHL_Init;
// initialization
// words must be sorted alphabetically
begin
  // set data for linear mode
  CHL_LM_Begin := SplitString('//,{,(*,''',',');
  CHL_LM_End := SplitString(chr(13)+',},*),''',',');
  CHL_LM_Color := SplitString('#00009F,#00009F,#00009F,#FF0000',',');
  CHL_LM_Italic := SplitString('0,0,0,1',',');
  // keyword data
  CHL_KeyWord1 := SplitString('absolute,abstract,and,array,as,asm,assembler,automated,begin,case,'+
    'cdecl,class,const,constructor,destructor,dispid,dispinterface,div,do,downto,dynamic,'+
    'else,end,except,export,exports,external,far,file,finalization,finally,for,forward,'+
    'function,goto,if,implementation,in,inherited,initialization,inline,interface,is,'+
    'label,library,message,mod,near,nil,not,object,of,or,out,override,packed,pascal,'+
    'private,procedure,program,property,protected,public,published,raise,record,register,'+
    'repeat,resourcestring,safecall,set,shl,shr,stdcall,then,threadvar,to,try,type,unit,'+
    'until,uses,var,virtual,while,with,xor',',');
  CHL_KeyWord2 := SplitString('ansichar,ansistring,array,boolean,byte,bytebool,cardinal,'+
    'char,currency,double,false,fixedint,fixeduint,int64,integer,longbool,longint,'+
    'longword,nativeint,nativeuint,pansichar,pansistring,pboolean,pbyte,pcardinal,'+
    'pchar,pcurrency,pdouble,pfixedint,pfixeduint,pint64,pinteger,plongbool,plongint,'+
    'plongword,pnativeint,pnativeuint,pointer,ppointer,prawbytestring,pshortint,'+
    'pshortstring,psingle,psmallint,pstring,puint64,punicodestring,pvariant,'+
    'pwidechar,pwidestring,pword,pwordbool,rawbytestring,real,record,shortint,'+
    'shortstring,single,smallint,string,tdoublerec,text,true,tsinglerec,type,'+
    'uint64,unicodestring,variant,widechar,widestring,word,wordbool',',' );
end;
Code language: PHP (php)

The list of standard types includes all types that are available in Delphi, but there are far fewer in My Visual Database. If we reduce this list, leaving only the necessary values, then the performance of the backlight module will be even higher.

What’s next?

Attentive reader has noticed that on the picture with the editing form there is a “Format code” button, the action of which deserves a separate description. This button allows you to align indents, wrap lines, and perform other actions that will make the source code look better. But I will talk about this in the next article.