MD5 encrypted Suppression lists

Discussion in 'Noob Central' started by CyVEX, Mar 22, 2012.

  1. CyVEX

    CyVEX New Member

    Joined:
    Mar 15, 2012
    Messages:
    45
    Likes Received:
    6
    Trophy Points:
    0
    Hey guys notice Crush are using these, I've not had the pleasure of having to use them before,

    How can you implement them to use? I'm not sure I got anything that will handle them :thumpdown:
     
  2. VelocitySoftware

    VelocitySoftware Member

    Joined:
    Sep 2, 2011
    Messages:
    66
    Likes Received:
    2
    Trophy Points:
    8
    Home Page:
    You would need to generate a md5 hash for each email in your list. Make sure you do lowercase on all your emails on the list before you do the hash. Then with the md5 list of your emails compare it to the suppression list md5 and remove any records that match with matching md5 hashes
     
  3. jellyfish

    jellyfish VIP

    Joined:
    Mar 15, 2011
    Messages:
    231
    Likes Received:
    3
    Trophy Points:
    0
    you can do it pretty quickly with mysql and a simple table with all your subscribers, really you can do all kinds of list management queries with mysql...
     
  4. CyVEX

    CyVEX New Member

    Joined:
    Mar 15, 2012
    Messages:
    45
    Likes Received:
    6
    Trophy Points:
    0
    Thanks for the tips guys I ended up just using a tool to encrypt each email then added it next to the original in a CSV, but still having issues filtering, my list manager software only has the ability to filter and remove emails not other fields.

    There has to be a more time affective way if there are a lot of companies using these encrypted files?
     
  5. JohnFarrell

    JohnFarrell VIP

    Joined:
    Apr 13, 2011
    Messages:
    828
    Likes Received:
    35
    Trophy Points:
    0
    If you use Linux it's pretty easy.

    <== begin list.txt contents ==>

    [email protected]
    [email protected]
    [email protected]
    [email protected]
    [email protected]
    [email protected]

    <== end list.txt contents ==>

    <== begin supfile.txt contents ==>

    3239824574d67c94eb34a50ff9e30e61
    6639439681b7351baad53456119cc2e6
    2d540ec7750438783ca9a3125d21aa38

    <== end supfile.txt contents ==>

    <== what the supfile looks like unhashed ==>

    [email protected]
    [email protected]
    [email protected]

    <== end ==>

    Upload your suppression file.

    First get a file that correlates the email address to the md5 hash.

    That creates a file called md5list.txt that looks like the following.

    3239824574d67c94eb34a50ff9e30e61 [email protected]
    6639439681b7351baad53456119cc2e6 [email protected]
    2d540ec7750438783ca9a3125d21aa38 [email protected]
    4a560fd80020ac68e74a0e7d80efd105 [email protected]
    2f32bf01cbed3d2f41cb09953dfddf94 [email protected]
    0d4f8c9fca2eb2d1801bda5b6446cf78 [email protected]


    okay, now we need to filter it.

    That creates a new file called filtered-list.txt that will contain all of the email addresses not contained in the suppression file.

    On a side note using MD5 for hashing emails to suppress is dangerous and shouldn't be done. If the person handing you the suppression file claims it's all upper case, but it's really all lower case you're fucked. If they have some sort of fuck up and they hash the address with a random cap or a random lower case (depending on if it's all upper case or all lower case) you're fucked. If they think it some how protects email addresses they're wrong. It does nothing, but give mailers extra hoops to jump through.
     
    Last edited: Mar 23, 2012
  6. mx10

    mx10 VIP

    Joined:
    Mar 30, 2011
    Messages:
    535
    Likes Received:
    18
    Trophy Points:
    18
    i agree md5 is a really bad idea.

    this is a C student industry.

    any barrier you put between someone and something important, no matter how trivial, is going to get fucked up.

    the advertiser will fuck up the md5, the network will, or the mailer will, or more likely than not: all 3.

    somehow the industry managed to survive for years with plain text suppression files. anyone who mails suppression files is obviously an idiot & a criminal and theres no reasonable precaution you can take against that kind of brazenness.
     
  7. DAgent

    DAgent Moderator

    Joined:
    Mar 14, 2011
    Messages:
    452
    Likes Received:
    115
    Trophy Points:
    43
    Home Page:
    +rep, useful contribution. More than 20 chars
     
  8. sjinks

    sjinks VIP

    Joined:
    Apr 27, 2011
    Messages:
    301
    Likes Received:
    31
    Trophy Points:
    28
    Location:
    Sevastopol, Ukraine
    Home Page:
    I understand that my reply is a bit late, but your hashsums are incorrect - yet another reason why it is easy to screw up with MD5 suppression files.

    MD5 hash of [email protected] will be dfeafc750cecde54f9a4775f5713bf01

    Code:
    $ for i in [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]; do php -r "echo '$i', ' ', md5('$i'), \"\n\";"; done
    [email protected] dfeafc750cecde54f9a4775f5713bf01
    [email protected] 963c64f2a0dc7bae3214ee63b9094c0f
    [email protected] bfc9b626df98f4a84e022204330c284d
    [email protected] c7f30c875dd0915003ac21f911ea8800
    [email protected] f4af098123cfa34d8a29cdef1da077e4
    [email protected] e9000542f74bf6ecc71adcf0f9ab55df
    
     
  9. sjinks

    sjinks VIP

    Joined:
    Apr 27, 2011
    Messages:
    301
    Likes Received:
    31
    Trophy Points:
    28
    Location:
    Sevastopol, Ukraine
    Home Page:
    Here's the bug:

    'echo' in Linux prints trailing LF (line feed) character which will obviously change the value of the hash. The correct way is to use

    Code:
    echo -n $i | md5sum
    
     
  10. JohnFarrell

    JohnFarrell VIP

    Joined:
    Apr 13, 2011
    Messages:
    828
    Likes Received:
    35
    Trophy Points:
    0
    oh snap, you're 100% correct. I forgot about the new line lawlz. this just reiterates my point of md5 for suppression files sucking ass.
     
  11. mx10

    mx10 VIP

    Joined:
    Mar 30, 2011
    Messages:
    535
    Likes Received:
    18
    Trophy Points:
    18
    this thread couldn't be more perfect
     
  12. CyVEX

    CyVEX New Member

    Joined:
    Mar 15, 2012
    Messages:
    45
    Likes Received:
    6
    Trophy Points:
    0
    Thanks for all the quality help guys!
     
    Last edited: Mar 25, 2012
  13. DKPMO

    DKPMO VIP

    Joined:
    Mar 31, 2011
    Messages:
    1,452
    Likes Received:
    68
    Trophy Points:
    48
    Location:
    Elaborate Underground Base
    This is all true. However these days it is hard to avoid having to support MD5 in one way or another.

    If you really have to ensure MD5 works correctly with others in the chain you just have to write a test suite and have your partners periodically run it on known hashes...

    You cannot force everyone to run such tests but that's really the only way to reduce breakage is to do something like that...
     

Share This Page