When attachments are added via the New Comment page and sent out via an email, the attachment will be added both to the email and at the Case level. When this is performed, the Attachments (and Google Docs & Attachments) related list in Lightning reads this as two separate attachments.

As Salesforce is planning to deprecate attachments within the Lightning Experience, the suggested solution is to adjust your Email to Case Premium to utilize Salesforce Files. Some more information on this switch and the configuration options available can be found here.