Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to configure Local PV - backed by Block Device with XFS? #2917

Open
briankmatheson opened this issue Feb 18, 2020 · 1 comment
Open

How to configure Local PV - backed by Block Device with XFS? #2917

briankmatheson opened this issue Feb 18, 2020 · 1 comment

Comments

@briankmatheson
Copy link

@briankmatheson briankmatheson commented Feb 18, 2020

Description

Expected Behavior

it should just work ;^) But if not it needs to feed the errored state back somehow. 
in this case it probably needs to say that the disk is already mounted. 

## Current Behavior
<!-- Tell us what happens instead -->
Repro'ed:

 bmath@timesink:~/maya/gke/test86$ cat cassandra-localpv-storageClass.yaml
 apiVersion: storage.k8s.io/v1
 kind: StorageClass
 metadata:
   name: cassandra-device
   annotations:
     openebs.io/cas-type: local
     cas.openebs.io/config: |
       - name: FSType
         value: xfs
 provisioner: openebs.io/local
 volumeBindingMode: WaitForFirstConsumer
 reclaimPolicy: Delete
 
 bmath@timesink:~/maya/gke/test86$ kubectl apply -f cassandra-
 cassandra-localpv-storageClass.yaml  cassandra-stress-job.yaml            

 bmath@timesink:~/maya/gke/test86$ kubectl apply -f cassandra-localpv-storageClass.yaml 
 storageclass.storage.k8s.io/cassandra-device created

 bmath@timesink:~/maya/gke/test86$ ./install_cassandra
 NAME:   cassandra
 LAST DEPLOYED: Mon Feb 17 14:08:47 2020
 NAMESPACE: cassandra
 STATUS: DEPLOYED
 
 RESOURCES:
 ==> v1/Pod(related)
 NAME         READY  STATUS             RESTARTS  AGE
 cassandra-0  0/1    ContainerCreating  0         0s
 
 ==> v1/Service
 NAME       TYPE       CLUSTER-IP  EXTERNAL-IP  PORT(S)                                       AGE
 cassandra  ClusterIP  None        <none>       7000/TCP,7001/TCP,7199/TCP,9042/TCP,9160/TCP  0s
 
 ==> v1/StatefulSet
 NAME       READY  AGE
 cassandra  0/3    0s
 
 
 NOTES:
 Cassandra CQL can be accessed via port 9042 on the following DNS name from within your cluster:
 Cassandra Thrift can be accessed via port 9160 on the following DNS name from within your cluster:
 
 If you want to connect to the remote instance with your local Cassandra CQL cli. To forward the API port to localhost:9042 run the following:
 - kubectl port-forward --namespace cassandra $(kubectl get pods --namespace cassandra -l app=cassandra,release=cassandra -o jsonpath='{ .items[0].metadata.name }') 9042:9042
 
 If you want to connect to the Cassandra CQL run the following:
 - kubectl port-forward --namespace cassandra $(kubectl get pods --namespace cassandra -l "app=cassandra,release=cassandra" -o jsonpath="{.items[0].metadata.name}") 9042:9042
   echo cqlsh 127.0.0.1 9042
 
 You can also see the cluster status by run the following:
 - kubectl exec -it --namespace cassandra $(kubectl get pods --namespace cassandra -l app=cassandra,release=cassandra -o jsonpath='{.items[0].metadata.name}') nodetool status
 
 To tail the logs for the Cassandra pod run the following:
 - kubectl logs -f --namespace cassandra $(kubectl get pods --namespace cassandra -l app=cassandra,release=cassandra -o jsonpath='{ .items[0].metadata.name }')
 
 bmath@timesink:~/maya/gke/test86$ kubectl get bd -A
 NAMESPACE   NAME                                           NODENAME                                      SIZE           CLAIMSTATE   STATUS   AGE
 openebs     blockdevice-1a89fd1f4bc1fedbfb268e91812b3ad3   gke-bmath-test86-default-pool-2134831e-703r   402653184000   Unclaimed    Active   3h54m
 openebs     blockdevice-8e45b547a555278739ab6db31d9ed7a9   gke-bmath-test86-default-pool-de467c51-pqtg   402653184000   Unclaimed    Active   3h54m
 openebs     blockdevice-e79e3a47c2e5ce23074a8df30d735e88   gke-bmath-test86-default-pool-fb0d6073-q14n   402653184000   Unclaimed    Active   3h54m

 bmath@timesink:~/maya/gke/test86$ kubectl get pvc -n cassandra 
 NAME               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS       AGE
 data-cassandra-0   Bound    pvc-b138c5ac-519b-11ea-b47f-42010a800196   10Gi       RWO            cassandra-device   3h29m
 data-cassandra-1   Bound    pvc-fcb08dfa-519b-11ea-b47f-42010a800196   10Gi       RWO            cassandra-device   3h27m
 data-cassandra-2   Bound    pvc-44c8250c-519c-11ea-b47f-42010a800196   10Gi       RWO            cassandra-device   3h25m


## Possible Solution
<!-- Suggest a fix or reason for this bug -->

Produce an actionable error appearing maybe in describe sc events list or 
some other likely place to check, and prevent the pod from just running on the root fs. 
@kmova kmova changed the title undetected failed lpv device mount Unable to use a device for Local PV Feb 18, 2020
@kmova
Copy link
Member

@kmova kmova commented Feb 18, 2020

The storage class for host path device is incorrectly configured. The docs only have an example for Local PV hostpath.

The correct way to configure a StorageClass for Local PV device - and format it with XFS is as follows:

  • Make sure the block devices are unformatted and unmounted. If they are, unmount and clear the filesystem information using wipefs -f -a /dev/sdb

  • Restart the NDM pods

  • Launch the storage class as follows:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: cassandra-device
  annotations:
    openebs.io/cas-type: local
    cas.openebs.io/config: |
      - name: StorageType
        value: device
      - name: FSType
        value: xfs
provisioner: openebs.io/local
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete
  • Launch the PVC with the above storage class.
    After the PVC is bound to a PV, you should see a BDC and BD in claimed state like below:
kubectl get pvc -n cassandra
NAME               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS       AGE
data-cassandra-0   Bound    pvc-4cf5c983-524d-11ea-b1e3-42010a80009a   10Gi       RWO            cassandra-device   47m
data-cassandra-1   Bound    pvc-96192219-524d-11ea-b1e3-42010a80009a   10Gi       RWO            cassandra-device   45m
data-cassandra-2   Bound    pvc-d324ed26-524d-11ea-b1e3-42010a80009a   10Gi       RWO            cassandra-device   43m  
kubectl get bdc -n openebs
NAME                                           BLOCKDEVICENAME                                PHASE   AGE
bdc-pvc-4cf5c983-524d-11ea-b1e3-42010a80009a   blockdevice-06ea8dae1b8f7aab3c138092f6e8ef55   Bound   48m
bdc-pvc-96192219-524d-11ea-b1e3-42010a80009a   blockdevice-edd9420ab63aafe982e6e90fb2036d17   Bound   45m
bdc-pvc-d324ed26-524d-11ea-b1e3-42010a80009a   blockdevice-59891e69ee7e20ca1d6216a84671d9a2   Bound   44m

When you login to the node, and do udevadm info /dev/sdb - you should see the block devices formatted with XFS filesystem

@kmova kmova changed the title Unable to use a device for Local PV How to configure Local PV - backed by Block Device with XFS? Feb 18, 2020
@kmova kmova added the docs label Feb 18, 2020
@kmova kmova self-assigned this Feb 18, 2020
@kmova kmova added this to Pre-commits and Designs - Due: Feb 29th 2020 in 1.8 Release Tracker - Due Mar 15th. Feb 18, 2020
@kmova kmova added this to the 1.8 milestone Feb 18, 2020
@kmova kmova moved this from Pre-commits and Designs - Due: Feb 29th 2020 to RC2 - Due Mar 10, 2020 in 1.8 Release Tracker - Due Mar 15th. Mar 6, 2020
@mynktl mynktl moved this from RC2 - Due Mar 10, 2020 to Release Items in 1.8 Release Tracker - Due Mar 15th. Mar 11, 2020
@payes payes moved this from Release Items to Pushed to Next release due to WIP in 1.8 Release Tracker - Due Mar 15th. Mar 16, 2020
@kmova kmova modified the milestones: 1.8, 1.9 Mar 18, 2020
@payes payes added this to Pre-commits and Designs - Due: Mar 31st 2020 in 1.9 Release Tracker - Due Apr 15th. Mar 18, 2020
@utkarshmani1997 utkarshmani1997 moved this from Pre-commits and Designs - Due: Mar 31st 2020 to RC1 - Due: Apr 5 2020 in 1.9 Release Tracker - Due Apr 15th. Apr 2, 2020
@utkarshmani1997 utkarshmani1997 moved this from RC1 - Due: Apr 5 2020 to RC2 - Due Apr 10, 2020 in 1.9 Release Tracker - Due Apr 15th. Apr 3, 2020
@utkarshmani1997 utkarshmani1997 moved this from RC2 - Due Apr 10, 2020 to Release Items in 1.9 Release Tracker - Due Apr 15th. Apr 13, 2020
@utkarshmani1997 utkarshmani1997 moved this from Release Items to Pushed to Next release due to WIP in 1.9 Release Tracker - Due Apr 15th. Apr 15, 2020
@kmova kmova added this to Pre-commits and Designs - Due: Apr 30 2020 in 1.10 Release Tracker - Due May 15th. Apr 22, 2020
@kmova kmova modified the milestones: 1.9, 1.10 Apr 22, 2020
@kmova kmova moved this from Pre-commits and Designs - Due: Apr 30 2020 to RC1 - Due: May 5 2020 in 1.10 Release Tracker - Due May 15th. May 4, 2020
@kmova kmova moved this from RC1 - Due: May 5 2020 to RC2 - Due May 10, 2020 in 1.10 Release Tracker - Due May 15th. May 6, 2020
@utkarshmani1997 utkarshmani1997 moved this from RC2 - Due May 10, 2020 to Release Items in 1.10 Release Tracker - Due May 15th. May 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
1.9 Release Tracker - Due Apr 15th.
  
Pushed to Next release due to WIP
1.8 Release Tracker - Due Mar 15th.
  
Pushed to Next release due to WIP
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.