To Disable the IPS or not?

[Gitsnik]

Something interesting just came up when I was talking to a friend of mine - we were discussing a penetration for PCI compliance, and the topic of IPS came up.

The lengthy argument was profuse and emphatic, but the same basic question (I feel) has to be asked:

During a PCI test (or any other), should one request the client turn off the IPS*? The two main arguments that I can see are:

No. The attacker wouldn't be able to get that turned off, why should you.
Yes. What if the attacker gets lucky and finds the IPS on the day it's failed - at least this way you can ensure you are not vulnerable.

So I put it to the rest of you, should the IPS be turned off for the pentester, or should it be left on?

*Turning it into IDS mode for that IP would be acceptable, the concern is the prevention part of the IPS.

[streaker69]

Why not try it both ways?

[lupin]

Id say it depends on the purpose of the test. The reason why a client generally gets a test done is to assess the security of some part of their infrastructure. These tests never cover everything, there is always some limitation in scope defined by what systems the client is most interested in assessing and why.

So if one of the purposes of the test is to assess the security of the network as is, then leaving the IPS on would be good. If the purpose of the test is to determine how well an application stands up to attack without the benefit of protective devices (in case they fail for example), then turn it off. If you want to assess the effectiveness of those protective devices, then you may need to perform tests with it on AND with it off (for comparison).

So, basically I think this should be decided upon as part of a scoping discussion with the client based on their requirements for the test.

[Thorn]

Lupin hit on the head. It's a scope issue. If you're doing a black box or gray box test, then the IPS should stay on. If it's a white box, then the state of the IPS would be negotiable, as a talented insider may have already disabled it or done an end run around it.

If it's a red team test, it's probably off already.

[Gitsnik]

Scoping for this job is basically "Ensure we are PCI compliant" which is vague, but ok. I actually took the side of the argument that the IPS is there and in place, so the test should be done with it ticking away. Even though an attacker may never get that opportunity, however, I was almost convinced that it should be turned off for maximum effect.

The question wasn't really about scoping, though I can see how important it is, it was more - at least for me - a question of how many defensive measures should one request be taken down (if any).

The mention of the red team is good, because I was going to say "We wouldn't ask for a firewall to be turned off" - but then again one has the chance to do a "disgruntled employee" pentest for this case.

Question still stands, I'm not comfortable that anyone has an opinion either way.

Originally Posted by lupin

Id say it depends on the purpose of the test. The reason why a client generally gets a test done is to assess the security of some part of their infrastructure. These tests never cover everything, there is always some limitation in scope defined by what systems the client is most interested in assessing and why.

So if one of the purposes of the test is to assess the security of the network as is, then leaving the IPS on would be good. If the purpose of the test is to determine how well an application stands up to attack without the benefit of protective devices (in case they fail for example), then turn it off. If you want to assess the effectiveness of those protective devices, then you may need to perform tests with it on AND with it off (for comparison).

So, basically I think this should be decided upon as part of a scoping discussion with the client based on their requirements for the test.

[lupin]

Originally Posted by Gitsnik

Scoping for this job is basically "Ensure we are PCI compliant" which is vague, but ok.

If the test is being done solely for the purpose of PCI compliance then the rational and reasoned process that could be used to determine scope when the client knows exactly what they want just goes out the window, and you are left with trying to interpret the PCI rules yourself or guess the way in which the assessor will interpret them.

I havent had any experience with PCI myself, but I have had experience with auditors and when you are doing anything just for the sake of compliance the degree of dilligence expected from you usually depends on how closely the auditor pays attention. The issue of whether to enable the IPS or not could even be moot in this case - the client may be able to just turn in a report with "Penetration Test" in the heading followed by pages of gibberish and get a tick in the box if the auditor is not checking closely.

My advice would be to study up on the Penetration Testing requirements from PCI and make sure you can justify whatever position you take based on your interpretation of those rules. If the rules dont mention anything related to disabling IPS then just do whatever makes you happy, because it probably won't matter to the client as long as they get their PCI compliance stamp.

My opinion, if you want it, and keeping in mind Im not PCI literate, is just leave the IPS on. It tests the security of the clients systems as it will normally be, it requires less effort from you and the client, and it doesn't leave the client more vulnerable during the window of the test. Personally though, the question becomes far less interesting for me when the decision is being made for compliance reasons rather than to provide a more nuanced security assessment for the client.

[Thorn]

Originally Posted by Gitsnik

Scoping for this job is basically "Ensure we are PCI compliant" which is vague, but ok.

Ugh. Good luck with that. PCI is all about checking off boxes on a checklist, and not seeing if the data is really protected.

According to the PCI DSS, none of the four levels that have electronic storage or transmittal need a pen test, but some are required to have er a "quarterly network scan", and the scan MUST be done by an Approved Scanning Vendor (ASV). If you (or your company) isn't a ASV, the client won't have met the requirement to be PCI compliant.

You might want to look at this:
https://www.pcisecuritystandards.org..._ASVs_v1-1.pdf

If the client is one of those that fall under the "self-assessment" validation levels, then all the need to do is one of the four (A, B, C, or D) Self-Assessment Questionnaires (SAQ). It's merely a matter of seeing if they have done certain things like having unique logins.

https://www.pcisecuritystandards.org...l#instructions

So the question about whether or not an IPS could or would be on or off is pretty moot. The fact of the matter is that, in all actuality, a pen test wouldn't be done for PCI compliance, but only a quarterly scan which only rises to the level of a low-end Vulnerability Assessment.

[freedom56]

The PCI DSS version 1.2 section 11.3 gives the only real guidance when it comes to pen tests for level 1 merchants/service providers. Basically it says perform internal and external pen tests at least annually and after any significant infrastructure or application upgrades. These tests are to include network and application layer pen tests.

The idea of PCI testing the security of the network through penetration testing implies that the test is done "as the network is currently configured". The company I work for does a lot (75% of our pen test business) of PCI compliance testing. PCI is more interested in testing your network as is without modification.

[=Z£Y$=]

Originally Posted by Gitsnik

During a PCI test (or any other), should one request the client turn off the IPS*? The two main arguments that I can see are:

No. The attacker wouldn't be able to get that turned off, why should you.
Yes. What if the attacker gets lucky and finds the IPS on the day it's failed - at least this way you can ensure you are not vulnerable.

To me, I will never actually advise a client to turn off the IPS/IDS system during their ASV scan. As this can put them at risk for the duration of their scan.

However i will do ask them to whitelist the IP addresses that the scan will originate from as the requirement states:-
"IDS/IPS should be configured to monitor and log but not to act against the
originating IP address of the ASV"

@freedom56 I think Gitsnik refers to the ASV scanning and not the pentest requirement of the PCI

[thorin]

According to the PCI Scanning Procedures for ASVs

13. Arrangements must be made to configure the intrusion detection system/intrusion prevention system (IDS/IPS) to accept the originating IP address of the ASV. If this is not possible, the scan should be originated in a location that prevents IDS/IPS interference