Add the following snippet in your post loop:

<?php
if ( function_exists( 'bp_follow_add_follow_button' ) ) :
	if ( bp_loggedin_user_id() && bp_loggedin_user_id() != get_the_author_meta( 'ID' ) ) {
		bp_follow_add_follow_button( array(
			'leader_id'   => get_the_author_meta( 'ID' ),
			'follower_id' => bp_loggedin_user_id()
		) );
	}
endif;
?>

Note: This only works in a post loop. If you're using this outside a post loop, you'll need to change get_the_author_meta( 'ID' ) to the user ID you want to follow.

Note 2: The code above means that the button only shows up if you are not the author of the current post.

You might also need to enqueue BP's stylesheet to view the button styles.