Tuberculosis (TB) is one of the infectious diseases that causes more deaths in low and middle-income countries. A low-cost method to diagnose TB consists in analyzing sputum smear samples through microscope observation. Manual identification and counting of bacilli is a very time consuming task and the sensitivity of the diagnosis depends on the availability of skilled technicians. We propose a computer vision technique based on a convolutional neural network (CNN) to automatically segment and count bacilli in sputum samples and predict the infection level.