In this method, a video camera records a long video of a structure of interest under typical operational or ambient vibrations. A region of interest in the video is defined around the structure. At least two stationary objects outside the region of interest must also be in the video frame as references for motion compensation. In-plane displacements of the object are extracted from the video for both the structure of interest and other regions of the video. The displacement signals from the structure of interest are compensated for camera motion, and then can be initially analyzed in the time domain or frequency domain by averaging the signals. Then any detailed analysis for the condition assessment of the structure can be carried out using the measured displacement signals of the structure of interest. These methods for monitoring changes in the displacement signal can include statistical pattern recognition techniques, one-class machine learning methods, analysis of non-linear features or similar damage detection algorithms.