Data Conversion to Standard data format in hierarchical Dirichlet process I'm trying to test the performance of posterior inference on a set of documents with hierarchical Dirichlet process for topic modeling.
   How can i convert  my data (document) to standard data format ?
such as this command :
[M] [term_1]:[count] [term_2]:[count] ...  [term_N]:[count]

where
[M] is the number of unique terms in the document
[count] associated with each term is how many times that term appeared 
in the document. 

 A: Seems like you need help with some basic parsing of a text. I'm not sure this is the right place to ask but I'll give you a quick answer here as I can't migrate questions between pages.  
Here is a simple solution in python:
#!/usr/bin/env python
import re
from collections import Counter

# change to text = input() to read from STDIN instead.           
text = """This is a text where there are several words and a few repeating
          terms. The text is nonsensical but this is not of interest."""


# this splits the text into lower case tokens and uses a 
# Counter to count the occurrences.
terms = Counter(re.split("[\s\.]+", text.lower().strip('.')))
M = len(terms)

# Print the data on the requested format
print(' '.join([str(M)] + ['{0}:{1}'.format(k, v) for k, v in terms.items()]))

This outputs:
19 a:2 of:1 terms:1 is:3 there:1 this:2 text:2 but:1 and:1 the:1 few:1 words:1 nonsensical:1 are:1 repeating:1 several:1 not:1 where:1 interest:1

The script is quite simple and depending on the amount of data you are processing it might not be sufficient. 
