Why CISA is Warning CISOs About a Breach at Sisense Krebs on Security

pThe US Cybersecurity and Infrastructure Security Agency CISA said today it is investigating a breach at business intelligence company Sisense whose products are designed to allow companies to view the status of multiple thirdparty online services in a single dashboard CISA urged all Sisense customers to reset any credentials and secrets that may have been shared with the company which is the same advice Sisense gave to its customers Wednesday eveningppppNew York City based Sisense has more than a thousand customers across a range of industry verticals including financial services telecommunications healthcare and higher education On April 10 Sisense Chief Information Security Officer Sangram Dash told customers the company had been made aware of reports that certain Sisense company information may have been made available on what we have been advised is a restricted access server not generally available on the internetppWe are taking this matter seriously and promptly commenced an investigation Dash continued We engaged industryleading experts to assist us with the investigation This matter has not resulted in an interruption to our business operations Out of an abundance of caution and while we continue to investigate we urge you to promptly rotate any credentials that you use within your Sisense applicationppIn its alert CISA said it was working with private industry partners to respond to a recent compromise discovered by independent security researchers involving SisenseppCISA is taking an active role in collaborating with private industry partners to respond to this incident especially as it relates to impacted critical infrastructure sector organizations the sparse alert reads We will provide updates as more information becomes availableppppSisense declined to comment when asked about the veracity of information shared by two trusted sources with close knowledge of the breach investigation Those sources said the breach appears to have started when the attackers somehow gained access to the companys Gitlab code repository and in that repository was a token or credential that gave the bad guys access to Sisenses Amazon S3 buckets in the cloudppCustomers can use Gitlab either as a solution that is hosted in the cloud at Gitlabcom or as a selfmanaged deployment KrebsOnSecurity understands that Sisense was using the selfmanaged version of GitlabppBoth sources said the attackers used the S3 access to copy and exfiltrate several terabytes worth of Sisense customer data which apparently included millions of access tokens email account passwords and even SSL certificatesppThe incident raises questions about whether Sisense was doing enough to protect sensitive data entrusted to it by customers such as whether the massive volume of stolen customer data was ever encrypted while at rest in these Amazon cloud serversppIt is clear however that unknown attackers now have all of the credentials that Sisense customers used in their dashboardsppThe breach also makes clear that Sisense is somewhat limited in the cleanup actions that it can take on behalf of customers because access tokens are essentially text files on your computer that allow you to stay logged in for extended periods of time sometimes indefinitely And depending on which service were talking about it may be possible for attackers to reuse those access tokens to authenticate as the victim without ever having to present valid credentialsppBeyond that it is largely up to Sisense customers to decide if and when they change passwords to the various thirdparty services that theyve previously entrusted to SisenseppEarlier today a public relations firm working with Sisense reached out to learn if KrebsOnSecurity planned to publish any further updates on their breach KrebsOnSecurity posted a screenshot of the CISOs customer email to both LinkedIn and Mastodon on Wednesday evening The PR rep said Sisense wanted to make sure they had an opportunity to comment before the story ranppBut when confronted with the details shared by my sources Sisense apparently changed its mindppAfter consulting with Sisense they have told me that they dont wish to respond the PR rep said in an emailed replyppUpdate 649 pm ET Added clarification that Sisense is using a selfhosted version of Gitlab not the cloud version managed by Gitlabcom ppAlso Sisenses CISO Dash just sent an update to customers directly The latest advice from the company is far more detailed and involves resetting a potentially large number of access tokens across multiple technologies including Microsoft Active Directory credentials GIT credentials web access tokens and any single signon SSO secrets or tokens ppThe full message from Dash to customers is belowppGood AfternoonppWe are following up on our prior communication of April 10 2024 regarding reports that certain Sisense company information may have been made available on a restricted access server As noted we are taking this matter seriously and our investigation remains ongoingppOur customers must reset any keys tokens or other credentials in their environment used within the Sisense applicationppSpecifically you should
Change Your Password Change all Sisenserelated passwords on httpmysisensecom
NonSSO
Replace the Secret in the Base Configuration Security section with your GUIDUUID
Reset passwords for all users in the Sisense application
Logout all users by running GET apiv1authenticationlogoutall under Admin user
Single SignOn SSO
If you use SSO JWT for the users authentication in Sisense you will need to update ssosharedsecret in Sisense and then use the newly generated value on the side of the SSO handler
We strongly recommend rotating the x509 certificate for your SSO SAML identity provider
If you utilize OpenID its imperative to rotate the client secret as well
Following these adjustments update the SSO settings in Sisense with the revised values
Logout all users by running GET apiv1authenticationlogoutall under Admin user
Customer Database Credentials Reset credentials in your database that were used in the Sisense application to ensure continuity of connection between the systems
Data Models Change all usernames and passwords in the database connection string in the data models
User Params If you are using the User Params feature reset them
Active DirectoryLDAP Change the username and user password of users whose authorization is used for AD synchronization
HTTP Authentication for GIT Rotate the credentials in every GIT project
B2D Customers Use the following API PATCH apiv2b2dconnection in the admin section to update the B2D connection
Infusion Apps Rotate the associated keys
Web Access Token Rotate all tokens
Custom Email Server Rotate associated credentials
Custom Code Reset any secrets that appear in custom code NotebooksppIf you need any assistance please submit a customer support ticket at httpscommunitysisensecomt5supportportalbdpSupportPortal and mark it as critical We have a dedicated response team on standby to assist with your requestsppAt Sisense we give paramount importance to security and are committed to our customers success Thank you for your partnership and commitment to our mutual securityppRegardsppSangram Dash
Chief Information Security Officerpp
This entry was posted on Thursday 11th of April 2024 0448 PM
ppIf they are telling people to rest credentialspprest resetppAlmost makes me wonder if some of these companies arent deliberately mishandling very sensitive information
This is conspiratorial but wouldnt it be logical for a foreign agency to sell a central dashboard to collect as many credentials as possibleppHow can they be that stupid to not encrypt every single bit of customer data not just security info but everything company names addresses etcppGood point there A honeypot in reverseppThe problem was someone checked a secret password API key TLS certificate private key etc into a GIT repository
Unfortunately software engineers make mistakes like this all of the time A lot of people think computer security is easy and therefore any breach is the result of incompetence or malice Usually that is not the case Here are the big three problemspp1 People including software engineers and operations peoplesys adminsdev ops people do not care
2 People do not know better
3 People do not want to put in the effort
4 People do not want to learn
5 People make mistakesppSolving these problems is going to be very hard and there are no easy answers ppAlso encryption is not a panacea because you still have to store a key to decrypt the data Hackers can and will exploit security vulnerabilities to get access to the decryption key Once they have the key they will get the encrypted data A good example of this I know of one company where people used to store encrypted TLS certificates in GIT About 50 of these encrypted certificates could be easily decrypted because people choose poor passwords The organization solved this problem by forcing people to store secrets securelyppYour items 14 are really quite the assertionppThe former leaving Amazon credentials in your Git archive is bad but forgivableppNot forgivable The thing with Git is that once provided information is hard to delete Even if deleted these credentials will be there in commit historyppThis is one more example of why you should never trust anyone with your sensitive information ppInstead selfhost as much as possible and always encrypt everythingppYour future self you thank you for thatppUnfortunately selfhosting is probably not any better In order for selfhosting to be better the person doing the work has to really understand the software being hosted and they have to consistently their infrastructure This means they have to install security patches promptly setup their firewalls correctly segment their network properly make sure they have configured the software correctly backup the data securely what happens if the house burns down etcppMy guess is most self hosters will have the same problems as professionals The professionals have the advantage of more resources and usually have better people An example is professional organizations are much more likely to notice they have been breachedppIt is possible to remove accidentallyincluded files by installing and using gitfilterrepo The full process of redactionsanitization is extremely difficult convoluted and dangerous destructive You will need to do several dryrun tests first make certain that you use inverted paths so that you are only deleting the offending files and make certain that there are backupsppAgain make certain that there are backups before doing anything Make certain that everyone else has pushed their changes to the repo before doing anything Make certain that everyone else has completely deleted their copy of the repo from their local disk before doing anythingppAt the end of the process the entire repository has been
purged of unwanted files
including any reference to their initial import
all of the commit hashes have been changedppPushing all the changes to your master repo is an absolute pain very convoluted Its easier to move the original repo to one side upload to a new repo and then have everyone else download from the new redactedsanitized repoppAssuming that you are keeping your own inhouse repo remember that your backup tapes etc have not been redactedsanitizedppAssuming that you are using github whoever anything pushed may linger there virtually for months To get that purged you need to contact github whoever staffppHow long will this take How big is your repo Mine was not huge so I have no baseline to offerppYou may find it much easier to get to a known good state remove the offending files and start a new git repo with this known good state as a new baseline YMMV regarding throwing away masses of historyppGood luckppThis is a bit surprising for an Israeli based company with their traditional focus on security but it sounds like Sisense has been in some turmoil now for a couple of years Perhaps this was a deliberate insider The company has been through a couple cycles of reorgs and mass layoffsppClear as mud thanks SisenseppIts still unclear if onprem instances are affected hoping to find out as were working with a customer on remediations and possibly taking precautions that arent necessaryppOnprem should not be at risk unlesssomeone like specifically shared secrets to the onprem with SisenseppOnprem should not be at risk unlesssomeone like specifically shared secrets to the onprem with SisenseppFor the record S3 encryption wouldve not helped at all Lets just get that clear Encryption in the cloud is more of a data access control rather than a data protection control If youre storing encrypted objects encryption prior to storage in S3 thats a different storyppDepends on how keys are stored although some cloud services namely Bitwarden and protonmail claim to have zeroknowledge of your keysppI took Nicks quoted remark about S3 without using encryption on top of it to mean encryption of the data prior to storage in S3 He knows what hes talking about so it seems to be the fairest interpretation To your point folks looking to protect data in S3 buckets should heed both your warning and NicksppTheres S3 serverside vs clientside SS has 3 keymgr options AWS C3 or SSEC
CS is not used as frequently Id assume theyre NOT encrypting beyond S3s SS
whether he knows what hes doing or not its the most common configuration and
S3 without using encryption on top taken at face value NOT encrypted beyond S3
If it were would this alleged breach even matter Theyd only have garbage stringsppIf youre running a pureAWS environment which I assume Sisense was then the specifics of the encryption are probably not relevant anyway If they had access to an IAM role via stolen credentials that role presumably had access to the encryption primitive as well as the S3 bucket Either they were using AES256 S3 encryption in which case no access is needed S3 KMS encryption in which case the role almost certainly had access to the KMS key as well as the S3 bucket or client side encryption In the case of client side encryption the client side is also executing in AWS and the client key is likely also stored in AWS somewhere either Secrets Manager or something like CloudHSM if youre getting fancy Its possible that the attacker had access to a role w S3 access but not Secrets Manager access but more likely that the compromised role had access to both So overall I think Weavers comment is a bit silly They were likely using encryption but it wasnt a relevant control against disclosure of the credentials that had access to the data the keyppThey were likely using encryption but it wasnt a relevant control against disclosure of the credentials that had access to the data the key
Well described Eggs in single basket shell thickness is no helpppThe ramifications of preforming that list are manifold and few but a systems admin are going to understand some of it Some of the Active Directory suggestions just as an example have the potential of shutting down the organization if there is a mistake or problem All one can say is JFC BTW when will CFOs and CEOs realize that we are at warppIf you are a competent system administrator or operations person you can easily preform all of those actionsppIt would be good to understand what is in S3 files
Total size indicates that most likely there is a lot of raw client data and keys can be only the tip of the icebergppFirst I know nothing on the subject When I saw the list of what customers should do I can only think that very few will perform all of these tasks in a timely manner I wonder how long it would take to have meetings and discussions in a corporation to discuss the impact to their users that these changes are going to have scheduling downtime in order to avoid loss of yearly bonuses as well as affecting the CEOs vacation scheduleppAll of the tasks boil down to changing passwords They are not hard to do and better organizations regularly rotate them ie change nonuser passwords periodicallyppI make it a rule to never provide thirdparty credentials to another service So far so goodppThat works until you want two services to work together Once you need that functionality you will have to give one service access to the otherppAt Sisense we give paramount importance to security and are committed to our customers success
Evidently notppIts always a good practice for organizations to stay vigilant maintain robust cybersecurity measures and promptly respond to any security advisories or warnings from trusted sources like CISAppIm not a security guy but this all makes sense Dare I say common sense But I am curious why is CISA trustedppCISA is the federal agency responsible for cybersecurity and infrastructure security in the United States under the remit of the Dept of Homeland Security
Unless you are a conspiracy theorist why wouldnt you trust them
And if not who would you trust to publish security advisories or cyber related warnings to the general publicppCISA is the federal agency responsible for cybersecurity and infrastructure security in the United States under the remit of the Dept of Homeland Security
Unless you are a conspiracy theorist why wouldnt you trust them
And if not who would you trust to publish security advisories or cyber related warnings to the general publicppOops This was a reply to Wannabe Techguy above Pls ignoreppComments are closedppMailing ListppSearch KrebsOnSecurityppRecent PostsppStory CategoriesppWhy So Many Top Hackers Hail from Russiap