Documenting PHP with Dexy

August 30, 2013   ·   By   ·   No Comments   ·   Posted in Examples

I just received a question about how to use Dexy to document procedural (non-OOP) PHP code. Here are some ways of using dexy to document PHP. These examples assume you already have familiarity with the contents of the Getting Started part of the guide.

Displaying Complete Files

Let’s take a simple example of an HTML file with embedded PHP:

<html>
<!-- @section "head" -->
<head>
<title>PHP Example</title>
</head>
<!-- @end -->
<?php

// examples taken from http://www.php.net/manual/en and modified

/// "assign-variables"
$b = $a = 5;
$a++;

/// "compare"
if ($a > $b) {
  echo "<p>a ($a) is bigger than b ($b)</p>";
  $b = $a;
}
/// @end
?>

<!-- @section "display-variables" -->
<?php
    echo "The value of a is $a."
?>

<!-- @end -->

</html>

We’ll use this example file in the subsequent examples. In order to display it above we used the pyg filter to apply syntax highlighting. This was specified in the dexy.yaml file as follows:

    - example.php|pyg:
        - pyg: { lexer: 'html+php' }

And included in this document like this:

{{ d["example.php|pyg"] }}

For more information, see the documentation page for dexy’s pyg filter.

Executing PHP

We can use dexy’s php filter to run this file through a php interpreter and see the result:

<html>
<!-- @section "head" -->
<head>
<title>PHP Example</title>
</head>
<!-- @end -->
<p>a (6) is bigger than b (5)</p>
<!-- @section "display-variables" -->
The value of a is 6.
<!-- @end -->

</html>

That was the contents displayed using a pre tag:

<pre>
{{ d["example.php|php"] | e }}
</pre>

We can also display the results with syntax highlighting:

<html>
<!-- @section "head" -->
<head>
<title>PHP Example</title>
</head>
<!-- @end -->
<p>a (6) is bigger than b (5)</p>
<!-- @section "display-variables" -->
The value of a is 6.
<!-- @end -->

</html>

{{ d["example.php|php|pyg"] }}

Here is how these are specified in dexy.yaml:

    - example.php|php
    - example.php|php|pyg:
        - pyg: { lexer: 'html' }

The contents of the resulting HTML file could also be displayed in HTML documentation using an iframe. This is done a lot in the Getting Started guide.

Sections of Files

To document what is going on in a file, we really need to be able to isolate sections of a file to discuss what is happening in that small section. We can do this using the idio filter in dexy, and fortunately this filter has recently been redone to support HTML-style comments as well as comments used in most programming languages, so you can mix and match these within an embedded PHP file. (You probably noticed these comments in the PHP file above.)

In the dexy.yaml file, we apply the idio filter to our php example file:

    - example.php|idio:
        - idio: { lexer: 'php', lexer-args: { startinline: True } }

Then, we can talk about just the head section in the HTML:

<head>
<title>PHP Example</title>
</head>

Or the assign-variables section:

$b = $a = 5;
$a++;

And then the section where we compare the variables:

if ($a > $b) {
  echo "<p>a ($a) is bigger than b ($b)</p>";
  $b = $a;
}

Here is the source of how these items were included in this document:

{{ d["example.php|idio"]["head"] }}
{{ d["example.php|idio"]["assign-variables"] }}
{{ d["example.php|idio"]["compare"] }}

HTML-style comments will be preserved as your php passes through the php filter (just be careful to leave a blank line after each closing ?>), so that you can also apply the idio filter to the output from the php filter:

    - example.php|php|idio:
        - idio: { lexer: 'html' }

In this way we can show the php:

<?php
    echo "The value of a is $a."
?>

And the resulting HTML:

The value of a is 6.

Here’s how we included each of these in this document:

{{ d["example.php|idio"]["display-variables"] }}
{{ d["example.php|php|idio"]["display-variables"] }}

Syntax Highlighting Sections

You might have noticed that the syntax highlighting wasn’t applied to the HTML content in the previous section, and we had to pass some custom arguments to the lexer to make things work for the PHP content. The PHP syntax highlighter in pygments is stateful and keeps track of whether it is in a HTML or PHP region of the source code file. However, because we pass our source code in in sections, the lexer isn’t able to keep track of the state.

This should be a solvable problem and you can track progress on it here, but for now you’ll have to tweak the settings if you want syntax highlighting, and please feel free to get in touch if you need help getting the output you want.

Deployment, Screenshots and Integration Testing

An additional approach which you can take to documenting PHP projects is to start a server locally (or on a virtual machine either locally or in the cloud) and run a headless web browser against that server to take screenshots and make assertions about the displayed content.

This approach is applicable to any web application, regardless of the underlying technology. It also allows you to document JavaScript components of your application. (You can also use the same pyg and idio filters to document JavaScript source files.)

The quickstart section of the Dexy Guide contains an example of bash scripts which start a local server, casper.js scripts which interact with the application running on that server, and then take screenshots and make assertions about the state of the application, as well as documentation which incorporates all these elements.

For example, here is a fully automated full-page screenshot of the polls app from the Django tutorial:

automated screenshot from Django tutorial polls app

SymPy and LaTeX

June 30, 2013   ·   By   ·   1 Comment   ·   Posted in Examples

SymPy is a computer algebra system for Python. This blog post will talk about how to use SymPy with Dexy to get automated mathematical LaTeX in your documents.

Read more…

Stargazing with Dexy

May 22, 2013   ·   By   ·   Comments Off   ·   Posted in Examples

Let’s check out the new Stargazer library for making beautiful PDF tables and see how to make it work with dexy. To use the R package we first load the library:

> library(stargazer)

Please cite as:

 Hlavac, Marek (2013). stargazer: LaTeX code for well-formatted regression and summary statistics tables.
 R package version 3.0.1. http://CRAN.R-project.org/package=stargazer

Here’s the basic demo:

> stargazer(attitude)

% Table created by StarGazer v.3.0.1 by Marek Hlavac, Harvard University. E-mail: hlavac at fas.harvard.edu
% Date and time: Wed, May 22, 2013 - 12:21:48 PM
\begin{table}[htb] \centering
  \caption{}
  \label{}
\footnotesize

\begin{tabular}{@{\extracolsep{5pt}}l c c c c c }
\\[-1.8ex]\hline
\hline \\[-1.8ex]
Statistic & \multicolumn{1}{c}{N} & \multicolumn{1}{c}{Mean} & \multicolumn{1}{c}{St. Dev.} & \multicolumn{1}{c}{Min} & \multicolumn{1}{c}{Max} \\
\hline \\[-1.8ex]
rating & 30 & 64.633 & 12.173 & 40 & 85 \\
complaints & 30 & 66.600 & 13.315 & 37 & 90 \\
privileges & 30 & 53.133 & 12.235 & 30 & 83 \\
learning & 30 & 56.367 & 11.737 & 34 & 75 \\
raises & 30 & 64.633 & 10.397 & 43 & 88 \\
critical & 30 & 74.767 & 9.895 & 49 & 92 \\
advance & 30 & 42.933 & 10.289 & 25 & 72 \\
\hline \\[-1.8ex]
\normalsize
\end{tabular}
\end{table}

This prints the LaTeX markup in our R session. The Stargazer docs say:

 ‘stargazer’ uses ‘cat()’ to output LaTeX code for the table. To
 allow for further processing of this output, ‘stargazer’ also
 returns the same output invisibly as a character string vector.
 You can include the produced tables in your paper by inserting
 ‘stargazer’ output into your publication's TeX source.

So, according to this there’s no built-in way to get stargazer to write its output directly to a file. One option would be to use R’s sink() to divert the output from stargazer:

> sink("output-from-sink.tex")
> stargazer(attitude)
> sink()

This gives us a file named output-from-sink.tex which we can include in documents.

That works fine, we can create as many separate files as we need. However, it might be nice to create a single file with multiple LaTeX snippets in it, referenced by name.

To do this we create a list and add named elements to the list containing output from running stargazer:

> latex.snippets <- list()
> latex.snippets["attitude"] <- paste(stargazer(attitude), collapse="\n")

% Table created by StarGazer v.3.0.1 by Marek Hlavac, Harvard University. E-mail: hlavac at fas.harvard.edu
% Date and time: Wed, May 22, 2013 - 12:21:49 PM
\begin{table}[htb] \centering
  \caption{}
  \label{}
\footnotesize

\begin{tabular}{@{\extracolsep{5pt}}l c c c c c }
\\[-1.8ex]\hline
\hline \\[-1.8ex]
Statistic & \multicolumn{1}{c}{N} & \multicolumn{1}{c}{Mean} & \multicolumn{1}{c}{St. Dev.} & \multicolumn{1}{c}{Min} & \multicolumn{1}{c}{Max} \\
\hline \\[-1.8ex]
rating & 30 & 64.633 & 12.173 & 40 & 85 \\
complaints & 30 & 66.600 & 13.315 & 37 & 90 \\
privileges & 30 & 53.133 & 12.235 & 30 & 83 \\
learning & 30 & 56.367 & 11.737 & 34 & 75 \\
raises & 30 & 64.633 & 10.397 & 43 & 88 \\
critical & 30 & 74.767 & 9.895 & 49 & 92 \\
advance & 30 & 42.933 & 10.289 & 25 & 72 \\
\hline \\[-1.8ex]
\normalsize
\end{tabular}
\end{table}

We’ll also generate the regression output example from Tal Galili’s blog post and save this to the list:

> latex.snippets["regression"] <- paste(regression.results, collapse="\n")

Then we export this list to a JSON file using the rjson library:

> library(rjson)
> latex.snippets.file <- file("stargazer-snippets.json", "w")
> writeLines(toJSON(latex.snippets), latex.snippets.file)
> close(latex.snippets.file)

Now let’s create a LaTeX document and make use of the snippets we have generated. Here’s the full source of the LaTeX document:

\documentclass{article}

\usepackage{graphicx}
\usepackage{color}
\usepackage{fancyvrb}
\usepackage{hyperref}

% Pygments syntax highlighting codes
<< pygments['pastie.tex'] >>

\begin{document}

Examples taken from \href{http://bit.ly/10TgaJp}{Tal Galili's blog post} about
stargazer.  This document is part of blog post \url{http://blog.dexy.it/523}.
Here's a pretty table:

<< d['output-from-sink.tex'] >>
<% set snippets = d['stargazer-snippets.json'].from_json() -%>
Here's the same table:

<< snippets.attitude >>

Here's a fancier table:

<< snippets.regression >>
Here is the source of this \LaTeX document:

<< d['example.tex|pyg|l'] >>

\end{document}

Here is how we configure the latex document to include the output from tables.R (you don’t need the ‘botoup’ filter unless you want to upload your PDF to Amazon S3):

    - example.tex|jinja|latex|botoup:
        - example.tex|pyg|l
        - tables.R|rout:
            - rout: {
                add-new-files: True
            }

Here’s part of the LaTeX document showing how we include the contents of the standalone file we created using sink():

Here's a pretty table:

<< d['output-from-sink.tex'] >>

To load our named list, we call the from_json() method to convert the JSON to a (python) dictionary:

<% set snippets = d['stargazer-snippets.json'].from_json() -%>

Then we can access the elements in this list as follows:

Here's the same table:

<< snippets.attitude >>

Here's a fancier table:

<< snippets.regression >>

We can use a dot to access the elements because of jinja. You can download the generated PDF to see what it looks like here.

That was two different ways to get Stargazer’s output into a file (for use with dexy).

Happy Pi Day

March 14, 2012   ·   By   ·   Comments Off   ·   Posted in Fun

It was pretty late last night when I finished my post about blogging with WordPress and Dexy, so I didn’t even notice it had turned into π day. This morning of course twitter reminded me, and in particular this caught my eye:

Read more…

Blogging with WordPress and Dexy

March 14, 2012   ·   By   ·   Comments Off   ·   Posted in Tutorials

For a developer, trying to blog about code is a frustrating process. WYSIWYG content editors mangle code and whitespace. Tools like gist and other code-snippet displays can help, but they also distance your code from your writing. With Dexy, we can write blog posts that incorporate code just where we want it, in the format we want it, and with confidence that the code we are writing is correct and easy to modify as we go.

In this tutorial we’ll walk through creating and then modifying a WordPress blog post. All the features mentioned in this blog post are available in Dexy 0.5.7. In order to post content to WordPress we will use its XMLRPC API, so before you try this yourself make sure you have enabled this option; it is disabled by default.

Read more…