Preprocessor for Verilog, SystemVerilog

DownLoad PDF Version Here

Contents

1. Introduction
2. Operation
3. Syntax, Keywords
4. Command Line options
5. Example of input file
6. Output of vpp.pl
7. Complete script: vpp.pl

Introduction

This is a generic Preprocessor / Macro Processor / Template Processor.

Syntax is different from m4, C preprocessor, Verilog preprocessor.

Since this is a generic text preprocessor, can be used for C, Verilog …

vpp.pl includes documentation of Syntax of input file. Jump to last section for the script.

Why to have another preprocessor for SystemVerilog?

SV is a verbose language. This preprocessor can be used to create template for reuse.

Vpp.pl has more features like for loop, can use perl data types, etc.

For example, UVM testbench skeleton can be made as a template; Use vpp.pl to generate files for agent, driver, monitor, … with uniform naming convention.

Operation

vpp.pl converts input files into perl code. Vpp.pl -p will show generated perl code.

The generated perl code is written as input to perl.

If there are no syntax errors, output is printed. If -o file option is used,

output is written to file instead of stdout.

If there are any syntax errors, error message may be about {, }, (, ), bad words, …,

Exact error in input file can’t be shown. Only location of line, file will be printed.

Syntax, Keywords

Syntax All Keywords Must start at begin of line and end with newline char

Leading spaces will be removed.

Exceptions: $get, $esc, macro calls – $user_defined_macro(…)

Keywords Meaning and Usage

$dnl Delete till newline character. Can be used to write comments

Example: This a line^ $dnl Read all comments. Leading spaces removed

This line will get joined to previous line, immediately after ‘^’

$macro Start of macro definition. Example:

$macro name $dnl Use ‘name’ for definition, Use $name for macro call

arg1 is $get($_[0])

arg2 is $get($_[1])

$end

Macro call: $name(‘arg1′,’arg2’)

$for Example:

$for my $i (1..10)

Iterator is $get($i)

$end

$set Perl Statement that ends with a ‘;’

Example:

$set $x = 5 $dnl Omit ending ‘;’

Value of $x is $get($x)

$begin Use this for perl statements that have ‘{‘

Example:

$begin if($_[0])

$begin while(1)

infinite loop

$end

$end

$begin elsif($_[1])

$begin if($_[2])

true part

$end

$begin else

false part

$end

$end

$end $macro, $for, $begin must end with matching $end

$esc Used to escape characters: ‘$’, ‘(‘, ‘)’

Example:

Use $name(‘$esc(left paran’) in place of $name(‘(left paran’)

Escape keywords: $esc$macro becomes $macro

$get Used to insert value of variables in plain text. See above examples

Can’t be used in lines with $macro, $for, $begin, $set, $end

Example: $get(‘string’) For ‘(‘, ‘)’ within $get, $esc must be used

Same rules of $get apply for user-defined-macro calls

Command line options

USAGE ./vpp.pl files [-o file] [-p] [-s] [-h]

-o Write/Over-Write to file

-p Show Generated Perl Code

-s Show Syntax for input file

-h Show Command line options

Example of input file

******* input.v begins after this line *******

1 abcdefghijklmnopqrstuvwxyz

$macro func

2 ab$get($_[0])cdefg$get (“$esc)”)hijklmnopqrstuvwxyz

$end

3 abcdefghi<<$esc$display>>jklmnopqrstuvwxyz

4 abcdefghijklmnopqrstuvwxyz

5 abcdefg<<$esc($esc)>>hijklmnopqrstuvwxyz

6 abcdefghijklmnopqrstuvwxyz

7 abcdefgh$esc$$$ijklmnopqrs$dnltuvwxyz

8 abcdefghijklmnopqrstuvwxy$esc$escz

9 abcdefghijklmnopqrstuvwxyz

$func(‘FIRST_ARGUMENT’)

$for(1..10) $dnl

$set $x = “argument”

$get($x) is $get($_)

$end

Printing all keyboard characters

a b c d e f g h i j k l m n o p q r s t u v w x y z

A B C D E F G H I J K L M N O P Q R S T U V W Z Y Z

1 2 3 4 5 6 7 8 9 0

! @ # $ % ^ & * ( )

` ~ – _ = +

[ { ] } \ |

; : ‘ ”

, < . > / ?

Output of vpp.pl

1 abcdefghijklmnopqrstuvwxyz

3 abcdefghi<<$display>>jklmnopqrstuvwxyz

4 abcdefghijklmnopqrstuvwxyz

5 abcdefg<<()>>hijklmnopqrstuvwxyz

6 abcdefghijklmnopqrstuvwxyz

7 abcdefgh$$$ijklmnopqrs8 abcdefghijklmnopqrstuvwxy$escz

9 abcdefghijklmnopqrstuvwxyz

2 abFIRST_ARGUMENTcdefg)hijklmnopqrstuvwxyz

argument is 1

argument is 2

argument is 3

argument is 4

argument is 5

argument is 6

argument is 7

argument is 8

argument is 9

argument is 10

Printing all keyboard characters

a b c d e f g h i j k l m n o p q r s t u v w x y z

A B C D E F G H I J K L M N O P Q R S T U V W Z Y Z

1 2 3 4 5 6 7 8 9 0

! @ # $ % ^ & * ( )

` ~ – _ = +

[ { ] } \ |

; : ‘ “

, < . > / ?

Complete Script: vpp.pl

******* vpp.pl begins after this line *******

#!/usr/bin/perl

# Author : A.G.Raja

# Description : m4 < This Preprocessor <= perl

############## PART1 : Parse Arguments #########################

my $outf = “STDOUT”; my $args = “”; my @infiles; my $genpl=0; my @cmd_out;

sub help { print STDERR “\n$_[0]\n\nUSAGE\t$0 files [-o file] [-p] [-s] [-h]\n\n”;

print STDERR “\t-o\tWrite/Over-Write to file\n”;

print STDERR “\t-p\tShow Generated Perl Code\n”;

print STDERR “\t-s\tShow Syntax for input file\n”;

print STDERR “\t-h\tShow Command line options\n”;

exit; }

sub parse_arguments {

for(my $in=0;$in<=$#ARGV;$in++) {

if($ARGV[$in] =~ /^\-o/) { $in++; $outf=$ARGV[$in]; }

elsif($ARGV[$in] =~ /^\-h/) { &help(“$0 converts input->perl_code->output”); }

elsif($ARGV[$in] =~ /^\-s/) { while(<DATA>) {print;} &help(“Command line options”); }

elsif($ARGV[$in] =~ /^\-p/) { $genpl=1; }

elsif($ARGV[$in] =~ /^\-/) { &help(“ERROR\tInvalid option ‘$ARGV[$in]’\n”); }

else { $args .= “$ARGV[$in] “; push(@infiles,$ARGV[$in]); } }

if($args !~ /\w+/) { &help(“ERROR\tNo input files specified\n”); }

if($outf =~ />\s*$/) { &help(“ERROR\tNo output file specified\n”); } }

############## PART2 : Text2Perl Conversion ####################

my $tt = “TEXT”; my $ee = “ESCAPE”; my $bt=”BEGIN$tt”; my $et=”END$tt”;

my $esn = $ee.”N”; my $esl = $ee.”L”; my $esr = $ee.”R”; my $esd = $ee.”D”;

my %hash; $hash{“get”}=”print”;

sub define_macro { my $n = shift; $n=~ s/\s//g; $hash{$n}=$n; }

sub macro_call {

my $out=””; my $func=””; my @l;

@l = split(/(\$\w+\s*\(|\))/,shift);

for(my $i=0;$i<=$#l;$i++) {

if($l[$i]=~/\$(\w+)/) {

$func=$hash{$1};

if($func ne “”) {

$out .= “$et $func(” ; $i++;

$out .= $l[$i] ; $i++;

$out .= “$l[$i]; $bt”; next; } }

$out .= $l[$i]; }

return $out; }

sub print_text {

my $in = shift;

if($in eq “”) { return “”; }

$in =~ s/\\/\\\\/g;

$in =~ s/\’/\\\’/g;

return “print ‘$in’;”; }

sub perl_code {

my ($prefix,$middle,$suffix,) = @_; my $ret;

$ret = “$et $prefix”;

if($middle =~ /(.*)$et($esn$bt)/) {

$ret .= “$1 $suffix$2”;

if($prefix eq “sub”) { &define_macro($1); } }

else {

if($prefix eq “sub”) { &define_macro($middle); }

$ret .= “$middle $suffix\n$bt”; }

return $ret; }

sub parse {

my $l = shift;

#$l=~s/\r//; # Uncomment This Line to run on Windows OS

$l=~s/[\$]esc\$/$esd/g;

$l=~s/[\$]esc\(/$esl/g;

$l=~s/[\$]esc\)/$esr/g;

$l=~s/[\$]dnl.*\n/$et$esn$bt/;

if($l=~/^\s*[\$]macro(.*)/) { $l = &perl_code(“sub”,$1,”{“); }

elsif($l=~/^\s*[\$]for(.*)/) { $l = &perl_code(“for”,$1,”{“); }

elsif($l=~/^\s*[\$]begin(.*)/) { $l = &perl_code(“” ,$1,”{“); }

elsif($l=~/^\s*[\$]set(.*)/) { $l = &perl_code(“” ,$1,”;”); }

elsif($l=~/^\s*[\$]end.*/) { $l = &perl_code(“}” ,””,”” ); }

else { $l = &macro_call($l); }

$l=~s/$esl/(/g;

$l=~s/$esr/)/g;

$l=~s/$esd/\$/g;

$l=~s/$esn/\n/g;

return $l; }

sub text2perl {

my $Text = shift;

my $out=””; my $pl=0; my @l;

@l = split(/($bt|$et|\n)/,$$Text);

for(@l) {

if($_ eq “$bt”) { $pl=0; }

elsif($_ eq “$et”) { $pl=1; }

elsif($_ eq “\n”) { $out .= $pl ? “\n” : “print \”\\n\”;\n”; }

else { $out .= $pl ? $_ : &print_text($_); } }

print “$out print \”Text2Perl Completed\”;”; }

############## PART3 : VPP – Generate Output ###################

sub vpp {

@cmd_out = `$0 -p $args | perl 2>&1`;

if($cmd_out[$#cmd_out]=~/Text2Perl Completed$/) {

$cmd_out[$#cmd_out]=~s/Text2Perl Completed$//;

if($outf eq “STDOUT”) { for(@cmd_out) {print;} exit; }

open(FILE,”>$outf”) or &help(“Can’t Write file ‘$outf'”);

for(@cmd_out) {print FILE;}

close(FILE);

return 1; }

return 0; }

my @b; my @e;

sub file_name {

my $ln = shift; my $act;

for(my $i=0;$i<=$#b;$i++) {

if( ($ln>$b[$i]) && ($ln<=$e[$i]+1) ) {

$act = $ln-$b[$i];

return “$act in ‘$infiles[$i]'”; } }

return “$ln”; }

sub report_errors {

my $line; my $curr=0;

for my $inf (@infiles) {

push(@b,$curr);

open(TEMP,”<$inf”) or last;

$line=0; while(<TEMP>) {$line++;}

close(TEMP);

$curr+=$line;

push(@e,$curr); }

for(@cmd_out) {

if(/\-\s*line\s*(\d+)/) {

$actual = &file_name($1);

s/(\-\s*line\s*)\d+/$1$actual/; }

print $_; } }

############## PART4 : Select Text2Perl / VPP ##################

&parse_arguments();

if(!$genpl) {

if(!&vpp()) { &report_errors(); }

exit; }

my $text=””;

for my $inf (@infiles) {

open(TEMP,”<$inf”) or last;

while(<TEMP>) { $text.=&parse($_); }

close(TEMP); }

&text2perl(\$text);

__DATA__

Syntax, Keywords, Usage Examples

Syntax All Keywords Must start at begin of line and end with newline char

Leading spaces will be removed.

Exceptions: $get, $esc, macro calls – $user_defined_macro(…)

Keywords Meaning and Usage

$dnl Delete till newline character. Can be used to write comments

Example: This a line^ $dnl Read all comments. Leading spaces removed

This line will get joined to previous line, immediately after ‘^’

$macro Start of macro definition. Example:

$macro name $dnl Use ‘name’ for definition, Use $name for macro call

arg1 is $get($_[0])

arg2 is $get($_[1])

$end

Macro call: $name(‘arg1′,’arg2’)

$for Example:

$for my $i (1..10)

Iterator is $get($i)

$end

$set Perl Statement that ends with a ‘;’

Example:

$set $x = 5 $dnl Omit ending ‘;’

Value of $x is $get($x)

$begin Use this for perl statements that have ‘{‘

Example:

$begin if($_[0])

$begin while(1)

infinite loop

$end

$end

$begin elsif($_[1])

$begin if($_[2])

true part

$end

$begin else

false part

$end

$end

$end $macro, $for, $begin must end with matching $end

$esc Used to escape characters: ‘$’, ‘(‘, ‘)’

Example:

Use $name(‘$esc(left paran’) in place of $name(‘(left paran’)

Escape keywords: $esc$macro becomes $macro

$get Used to insert value of variables in plain text. See above examples

Can’t be used in lines with $macro, $for, $begin, $set, $end

Example: $get(‘string’) For ‘(‘, ‘)’ within $get, $esc must be used

Same rules of $get apply for user-defined-macro calls

General Purpose Preprocessor in Perl

Here is a preprocessor in perl, primarily for Verilog, SystemVerilog, but may be used for any language.

There are programs with a lot of repetitive text. Language features like functions, objects can reduce typing.

Still, there is scope for putting it in lesser code using a text preprocessor.

Limitations in existing preprocessors prompted in writing a new one that addresses the issues

C preprocessor:

Every define has to be written in a single line.

Escape chars for special chars can make the macro unreadable

Very limited keywords. No loop operations

m4 preprocessor:

No escape characters

Difficult to write macros consisting special characters

Advantages of the new preprocessor:

Written in perl, no CPAN packages used, very portable

Very small – 50 lines of code; Only 7 keywords;

No escape characters. Can be used for any programming language.

Possible to emulate features of C-preprocessor as well as m4

Hook to insert perl keywords – shown in example at end of script.

More usage examples coming-up in later posts.

#!/usr/bin/perl

###### General Purpose Preprocessor (c) A.G.Raja ########

my @lines;

my $code;

my $t=0;

my $e;

sub tokenize {

  my $text = shift;

  my @spl  = split(/(\$get\s*\(\s*\w+)\s*\)/,$text);

  my $out  = “”;

  my $var  = “”;

  foreach my $s (@spl) {

    if($s =~ /\$get\s*\((.*)/) {

      $var = $1;

      if($var =~ /^\d+/) {

        $out .= “\$_[$var]”;

      } else { $out .= “\$$var”;

      }

    } else {

      push(@lines,$s);

      $out .= “\$lines[$#lines]”;

    }

  }

  return “\”$out\””;

}

while(<DATA>) {

  s/\$dnl.*\n//;

  if(!length($_)) {

  } elsif(/^\s*\$macro\s+(.*)/) {

    $code .= “\nsub $1 {\n  my \$ret;\n”;

    $t=1;

  } elsif(/^\s*\$endmacro\s*$/) {

    $code .= “return \$ret;\n}\n”;

    $t=0;

  } elsif(/^\s*\$begin\s+(.*)/) {

    $code .= “$1 {\n”;

  } elsif(/^\s*\$end\s*$/) {

    $code .= “}\n”;

  } elsif(/\$set\s+(\$\w+)\s*=(.*)/) {

    $code .= “\n$1 = \”\@{[ $2 ]}\”;\n”;

  } else {

    $e = &tokenize($_);

    if($t==1) {

      $code .= ”  \$ret .= $e;\n”;

    } else {

      $code .= “print $e;\n”;

    }

  }

}

eval $code or die print “\n\n### ERROR: PREPROCESSING FAILED ###\n\n”;

############   TEXT after DATA can be edited   ############

__DATA__

$macro func               $dnl ### This is Start of Macro Definition

$get(0) $dnl              $dnl ### $dnl will Delete-till New-Line

$begin foreach $x (@_)    $dnl ### $begin will append { to perl code

argument is $get(x)       $dnl ##### Substitute perl variable, here $x

$end                      $dnl ### $end will append } to perl code

$endmacro                 $dnl ### This is End of Macro Definition

$set $a = func(1,2,3,4,5) $dnl ### Assign return value of a Macro

aa $begin                 $dnl ### Spaces before keywords not allowed

gadfdasfdas

 $dnl this is a comment. newline+space will be printed here

$get(a)

$dnl ############################################

$dnl #### This Script translates TEXT to perl code and executes it.

$dnl #### Following are Keywords for TEXT to be preprocessed:

$dnl ####   $dnl $macro $endmacro $begin $end $set $get

$dnl #### Usage of the keywords are explained in above example.

$dnl #### $begin keyword can be exploited to get the power of perl.

$dnl ############################################