Speech translation is usually the pipeline task of automatic speech recognition (ASR), translation unit
segmentation and machine translation (MT). Segmenting the ASR output to translation units poses a challenge of balancing the translation quality and efficiency for real-time speech translation. In this paper, we firstly propose a parser-based semantic boundary detection method to detect all semantic boundaries based on our definition. To realize the translation of the semantic units, a word-boundary language model is secondly
proposed to improve the translation quality. Experiments on English to Chinese and Chinese to English speech translation have shown that the proposed method yields improved translation quality and lower latency, when compared to the conventional punctuated methods.