Skip to content

Cloud Storage

Cyderes has the ability to pull logs from GCP Cloud Storage to ingest into Chronicle

Log Ingestion

When using our GCP Cloud Storage integration, please note that only logs generated after the integration setup will be automatically ingested. For older logs existing prior to the setup, you will need to manually trigger notifications for ingestion. This means that logs generated post-integration will be seamlessly ingested, while any pre-existing older logs require manual notification activation for ingestion.

Cyderes Object Storage Best Practices

Best Practices

Creating a GCP Cloud Storage Bucket

  1. Create a new Cloud Storage bucket for the Cloud Storage bucket logs to be stored in. A pre-existing Cloud Storage bucket may be used. GCP Guide.

Access Configuration

IAM Service Account for bucket access

Depending on if you provide an IAM Service Account for bucket access or not will determine which steps to follow, please carefully note which steps are being followed depending which option is chosen.

with customer provided service account for bucket access

  1. Provide the following information to Cyderes for implementation:

    • GCP Cloud Storage Bucket Name
    • GCP Cloud Storage Bucket File Path
    • One Log Type for each GCP Cloud Storage Bucket Path
    • Cloud Storage Project Service Account
    • IAM Service Account for bucket access

    Note: The cloud storage project service account that is required should have the following format: service-1234567890@gs-project-accounts.iam.gserviceaccount.com

  2. Cyderes will use the information to provision a PubSub topic and a service account. Cyderes will then provide the service account email and the PubSub topic name to be used in the next steps.

  3. Set up events to go to the Cyderes-owned PubSub topic:

    • Use this guide to understand the command to run to enable notifications: Command Guide
    • The OBJECT_FINALIZE event type is needed for Cyderes to process logs from the storage bucket. If this is only managed by automation, the OBJECT_METADATA_UPDATE may be added to enable ease of reprocessing should the need arise. Docs to reference.
    • Example command from above: run gsutil notification create -s -t projects/cyderes-prod/topics/(TOPIC PROVIDED BY Cyderes) -f json -e OBJECT_FINALIZE gs://(YOUR BUCKET NAME)

without customer provided service account for bucket access

  1. Provide the following information to Cyderes for implementation:

    Note: The cloud storage project service account Cyderes requires should have the following format: service-1234567890@gs-project-accounts.iam.gserviceaccount.com

  2. Cyderes will use the information to provision a PubSub topic and a service account. Cyderes will then provide the service account email and the PubSub topic name to be used in the next steps.

  3. Grant permissions to retrieve objects from the Cloud Storage bucket to the Cyderes provided service account by following this GCP Guide. The required permissions are Storage Legacy Bucket Reader (roles/storage.legacyBucketReader) and Storage Object Viewer (roles/storage.objectViewer).

  4. Set up events to go to the Cyderes-owned PubSub topic:

    • Use this guide to understand the command to run to enable notifications: Command Guide
    • The OBJECT_FINALIZE event type is needed for Cyderes to process logs from the storage bucket. If this is only managed by automation, the OBJECT_METADATA_UPDATE may be added to enable ease of reprocessing should the need arise. Docs to reference.
    • Example command from above: run gsutil notification create -s -t projects/cyderes-prod/topics/(TOPIC PROVIDED BY Cyderes) -f json -e OBJECT_FINALIZE gs://(YOUR BUCKET NAME)